[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:37 dist-test-slave-45dx krb5kdc[15959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:37 dist-test-slave-45dx krb5kdc[15959](info): set up 2 sockets
Aug 06 00:48:37 dist-test-slave-45dx krb5kdc[15959](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 00:48:38.270352 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:48:38 dist-test-slave-45dx krb5kdc[15959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:48:38.401942 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:35027
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:35305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:35027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:38.529942 15979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:38.530267 15979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:38.530345 15979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:38.534848 15979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:38.534945 15979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:38.534986 15979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:38.535053 15979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:38.535094 15979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:38.541142 15979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:35305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:35027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:35027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:38.542788 15979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:38.544251 15979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:38.557252 15985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:38.557322 15989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:38.557245 15987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:38.558759 15979 server_base.cc:1034] running on GCE node
I20260806 00:48:38.559594 15979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:38.561261 15979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:38.562458 15979 hybrid_clock.cc:648] HybridClock initialized: now 1785977318562434 us; error 39 us; skew 500 ppm
Aug 06 00:48:38 dist-test-slave-45dx krb5kdc[15959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977318, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:38.565716 15979 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:48:38.566777 15979 webserver.cc:460] Webserver started at http://127.15.147.190:37641/ using document root <none> and password file <none>
I20260806 00:48:38.567673 15979 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:38.567775 15979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:38.568182 15979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:38.570937 15979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "eedbe6aecd5742088c65a11469852144"
format_stamp: "Formatted at 2026-08-06 00:48:38 on dist-test-slave-45dx"
server_key: "1c6fa4ea589880ab2a0518d537bddea2"
server_key_iv: "99d2c8ddeeaf1affee56dfcebea1823c"
server_key_version: "encryptionkey@0"
I20260806 00:48:38.571563 15979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "eedbe6aecd5742088c65a11469852144"
format_stamp: "Formatted at 2026-08-06 00:48:38 on dist-test-slave-45dx"
server_key: "1c6fa4ea589880ab2a0518d537bddea2"
server_key_iv: "99d2c8ddeeaf1affee56dfcebea1823c"
server_key_version: "encryptionkey@0"
I20260806 00:48:38.576678 15979 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 00:48:38.580178 16000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:38.581746 15979 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:48:38.581970 15979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "eedbe6aecd5742088c65a11469852144"
format_stamp: "Formatted at 2026-08-06 00:48:38 on dist-test-slave-45dx"
server_key: "1c6fa4ea589880ab2a0518d537bddea2"
server_key_iv: "99d2c8ddeeaf1affee56dfcebea1823c"
server_key_version: "encryptionkey@0"
I20260806 00:48:38.582093 15979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:38.604526 15979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:38.608085 15979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:38.608431 15979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:38.625124 15979 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:35027
I20260806 00:48:38.625269 16075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:35027 every 8 connection(s)
I20260806 00:48:38.626609 15979 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:38.631150 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 15979
I20260806 00:48:38.631282 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:48:38.631603 16078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:38.632970 15950 external_mini_cluster.cc:1428] Setting key 36458ec072b2aa81002f32ff1d97f488
I20260806 00:48:38.641681 16078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144: Bootstrap starting.
I20260806 00:48:38.646016 16078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:38.647660 16078 log.cc:824] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:38.650568 16078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144: No bootstrap required, opened a new log
Aug 06 00:48:38 dist-test-slave-45dx krb5kdc[15959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:38.656939 16078 raft_consensus.cc:348] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eedbe6aecd5742088c65a11469852144" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35027 } }
I20260806 00:48:38.657282 16078 raft_consensus.cc:374] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:38.657428 16078 raft_consensus.cc:729] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eedbe6aecd5742088c65a11469852144, State: Initialized, Role: FOLLOWER
I20260806 00:48:38.658262 16078 consensus_queue.cc:260] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eedbe6aecd5742088c65a11469852144" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35027 } }
I20260806 00:48:38.658468 16078 raft_consensus.cc:388] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:38.658614 16078 raft_consensus.cc:482] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:38.658762 16078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:38.659869 16078 raft_consensus.cc:504] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eedbe6aecd5742088c65a11469852144" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35027 } }
I20260806 00:48:38.660519 16078 leader_election.cc:302] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eedbe6aecd5742088c65a11469852144; no voters:
I20260806 00:48:38.660936 16078 leader_election.cc:288] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:38.661127 16087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:38.661502 16087 raft_consensus.cc:686] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144 [term 1 LEADER]: Becoming Leader. State: Replica: eedbe6aecd5742088c65a11469852144, State: Running, Role: LEADER
I20260806 00:48:38.661931 16087 consensus_queue.cc:237] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eedbe6aecd5742088c65a11469852144" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35027 } }
I20260806 00:48:38.662859 16078 sys_catalog.cc:564] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:38.663249 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:38.637695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46142 (local address 127.15.147.190:35027)
0806 00:48:38.638201 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:38.638225 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:38.639520 (+ 1295us) server_negotiation.cc:407] Connection header received
0806 00:48:38.644288 (+ 4768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:38.644370 (+ 82us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:38.644740 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:38.645054 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:38.647723 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:38.649502 (+ 1779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:38.650800 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:38.651320 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:38.651516 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:38.656129 (+ 4613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:38.656161 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:38.656174 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:38.656202 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:38.658501 (+ 2299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:38.659167 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:38.659177 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:38.659181 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:38.659294 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:38.659678 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:38.659681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:38.659683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:38.660422 (+ 739us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:38.660632 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:38.661454 (+ 822us) server_negotiation.cc:299] Negotiation successful
0806 00:48:38.661823 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":181,"threads_started":1}
I20260806 00:48:38.665697 16088 sys_catalog.cc:455] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eedbe6aecd5742088c65a11469852144" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eedbe6aecd5742088c65a11469852144" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35027 } } }
I20260806 00:48:38.665831 16088 sys_catalog.cc:458] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:38.666460 16089 sys_catalog.cc:455] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eedbe6aecd5742088c65a11469852144. Latest consensus state: current_term: 1 leader_uuid: "eedbe6aecd5742088c65a11469852144" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eedbe6aecd5742088c65a11469852144" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35027 } } }
I20260806 00:48:38.666575 16089 sys_catalog.cc:458] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:38.667577 16097 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:38.672204 16097 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:38.680984 16097 catalog_manager.cc:1347] Generated new cluster ID: ce88b1014c3f4647a3a7a64e7ee1a7e0
I20260806 00:48:38.681052 16097 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:38.724520 16097 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:38.725628 16097 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:38.735962 16097 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eedbe6aecd5742088c65a11469852144: Generated new TSK 0
I20260806 00:48:38.736790 16097 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
I20260806 00:48:38.775885 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:35027
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:35305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:38.938819 16111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:38.939066 16111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:38.939110 16111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:38.943114 16111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:38.943168 16111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:38.943269 16111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:48:38.948499 16111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:35305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:35027
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:38.949590 16111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:38.950599 16111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:38.962090 16120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:38.962209 16119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:38.962277 16111 server_base.cc:1034] running on GCE node
W20260806 00:48:38.962184 16123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:38.962841 16111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:38.963493 16111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:38.964651 16111 hybrid_clock.cc:648] HybridClock initialized: now 1785977318964617 us; error 47 us; skew 500 ppm
Aug 06 00:48:38 dist-test-slave-45dx krb5kdc[15959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977318, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:38.968040 16111 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:48:38.968856 16111 webserver.cc:460] Webserver started at http://127.15.147.129:45311/ using document root <none> and password file <none>
I20260806 00:48:38.969372 16111 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:38.969437 16111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:38.969595 16111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:38.971724 16111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "cf2e157791344dd3a2be895872ed188b"
format_stamp: "Formatted at 2026-08-06 00:48:38 on dist-test-slave-45dx"
server_key: "c16584c6e12224444c7ca89a9fb53fc0"
server_key_iv: "a4a19402b519c24d537483ea8fb41e0d"
server_key_version: "encryptionkey@0"
I20260806 00:48:38.972399 16111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "cf2e157791344dd3a2be895872ed188b"
format_stamp: "Formatted at 2026-08-06 00:48:38 on dist-test-slave-45dx"
server_key: "c16584c6e12224444c7ca89a9fb53fc0"
server_key_iv: "a4a19402b519c24d537483ea8fb41e0d"
server_key_version: "encryptionkey@0"
I20260806 00:48:38.976231 16111 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:48:38.978713 16133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:38.979985 16111 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:48:38.980078 16111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "cf2e157791344dd3a2be895872ed188b"
format_stamp: "Formatted at 2026-08-06 00:48:38 on dist-test-slave-45dx"
server_key: "c16584c6e12224444c7ca89a9fb53fc0"
server_key_iv: "a4a19402b519c24d537483ea8fb41e0d"
server_key_version: "encryptionkey@0"
I20260806 00:48:38.980157 16111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:38.993376 16111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:38.996373 16111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:38.996574 16111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:38.997298 16111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:48:38.998332 16111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:48:38.998380 16111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:38.998418 16111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:48:38.998433 16111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:39.018646 16111 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:34995
I20260806 00:48:39.018684 16293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:34995 every 8 connection(s)
I20260806 00:48:39.019675 16111 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:48:39.024742 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 16111
I20260806 00:48:39.024837 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:48:39.025127 15950 external_mini_cluster.cc:1428] Setting key eb4faeeccb080e6e665682b0b59f15ea
Aug 06 00:48:39 dist-test-slave-45dx krb5kdc[15959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977318, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:39.036917 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:39.022109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:42043 (local address 127.15.147.190:35027)
0806 00:48:39.022262 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:39.022266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:39.023254 (+ 988us) server_negotiation.cc:407] Connection header received
0806 00:48:39.025153 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:39.025158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:39.025233 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:39.025327 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:39.027260 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:39.028065 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:39.029445 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:39.029836 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:39.029958 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:39.033169 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:39.033172 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:39.033174 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:39.033198 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:39.034737 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:39.035262 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:39.035269 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:39.035271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:39.035355 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:39.035696 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:39.035699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:39.035701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:39.035904 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:39.036024 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:39.036607 (+ 583us) server_negotiation.cc:299] Negotiation successful
0806 00:48:39.036736 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 00:48:39.037806 16297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:39.022482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35027 (local address 127.15.147.129:42043)
0806 00:48:39.023089 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:39.023135 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:39.024942 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:39.025492 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:39.025499 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:39.026231 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:39.027100 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:39.027119 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:39.028185 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:39.028188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:39.029320 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:39.029330 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:39.029988 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:39.029990 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:39.030177 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:39.030920 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:39.030946 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:39.033002 (+ 2056us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:39.034869 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:39.034875 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:39.034889 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:39.035151 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:39.035447 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:39.035450 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:39.035452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:39.035593 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:39.036025 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:39.036032 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:39.036300 (+ 268us) client_negotiation.cc:769] Sending connection context
0806 00:48:39.036550 (+ 250us) client_negotiation.cc:240] Negotiation successful
0806 00:48:39.036887 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":201,"threads_started":1}
WARNING: no policy specified for kudu/127.15.147.130@KRBTEST.COM; defaulting to no policy
I20260806 00:48:39.039242 16294 heartbeater.cc:347] Connected to a master server at 127.15.147.190:35027
I20260806 00:48:39.039505 16294 heartbeater.cc:464] Registering TS with master...
I20260806 00:48:39.040086 16294 heartbeater.cc:511] Master 127.15.147.190:35027 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.130@KRBTEST.COM" created.
I20260806 00:48:39.042254 16027 ts_manager.cc:194] Registered new tserver with Master: cf2e157791344dd3a2be895872ed188b (127.15.147.129:34995)
I20260806 00:48:39.043712 16027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:42043
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
I20260806 00:48:39.059330 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:35027
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:35305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:39.216769 16303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:39.217006 16303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:39.217047 16303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:39.220701 16303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:39.220748 16303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:39.220847 16303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:48:39.225934 16303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:35305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:35027
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:39.226930 16303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:39.227927 16303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:39.238413 16312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:39.238477 16309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:39.238458 16310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:39.238672 16303 server_base.cc:1034] running on GCE node
I20260806 00:48:39.239035 16303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:39.239615 16303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:39.240772 16303 hybrid_clock.cc:648] HybridClock initialized: now 1785977319240759 us; error 23 us; skew 500 ppm
Aug 06 00:48:39 dist-test-slave-45dx krb5kdc[15959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977319, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:39.244093 16303 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:48:39.244989 16303 webserver.cc:460] Webserver started at http://127.15.147.130:37787/ using document root <none> and password file <none>
I20260806 00:48:39.245507 16303 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:39.245564 16303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:39.245810 16303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:39.247998 16303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "2cc3729bbc584524a37dfd5d6fdb1f4b"
format_stamp: "Formatted at 2026-08-06 00:48:39 on dist-test-slave-45dx"
server_key: "06f42317c44f030ab317f7328608f3b3"
server_key_iv: "6a4282952caf9fc48c0ebba872b4d542"
server_key_version: "encryptionkey@0"
I20260806 00:48:39.248656 16303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "2cc3729bbc584524a37dfd5d6fdb1f4b"
format_stamp: "Formatted at 2026-08-06 00:48:39 on dist-test-slave-45dx"
server_key: "06f42317c44f030ab317f7328608f3b3"
server_key_iv: "6a4282952caf9fc48c0ebba872b4d542"
server_key_version: "encryptionkey@0"
I20260806 00:48:39.253034 16303 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 00:48:39.255707 16323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:39.257126 16303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:48:39.257253 16303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "2cc3729bbc584524a37dfd5d6fdb1f4b"
format_stamp: "Formatted at 2026-08-06 00:48:39 on dist-test-slave-45dx"
server_key: "06f42317c44f030ab317f7328608f3b3"
server_key_iv: "6a4282952caf9fc48c0ebba872b4d542"
server_key_version: "encryptionkey@0"
I20260806 00:48:39.257375 16303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:39.276796 16303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:39.279850 16303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:39.280077 16303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:39.280936 16303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:48:39.281908 16303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:48:39.281966 16303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:39.282059 16303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:48:39.282114 16303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:39.298892 16303 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:38863
I20260806 00:48:39.298972 16486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:38863 every 8 connection(s)
I20260806 00:48:39.299886 16303 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:48:39.306671 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 16303
I20260806 00:48:39.306806 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:48:39.307085 15950 external_mini_cluster.cc:1428] Setting key 2cde093dee652920993ddd18ac22d999
Aug 06 00:48:39 dist-test-slave-45dx krb5kdc[15959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977319, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:39.316926 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:39.301900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:54305 (local address 127.15.147.190:35027)
0806 00:48:39.302049 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:39.302053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:39.302955 (+ 902us) server_negotiation.cc:407] Connection header received
0806 00:48:39.305081 (+ 2126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:39.305085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:39.305148 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:39.305249 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:39.306811 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:39.307441 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:39.309108 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:39.309343 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:39.309430 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:39.313093 (+ 3663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:39.313096 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:39.313098 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:39.313123 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:39.314638 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:39.315269 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:39.315272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:39.315273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:39.315345 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:39.315752 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:39.315754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:39.315755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:39.315935 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:39.316007 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:39.316708 (+ 701us) server_negotiation.cc:299] Negotiation successful
0806 00:48:39.316808 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:48:39.318426 16489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:39.302236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35027 (local address 127.15.147.130:54305)
0806 00:48:39.302793 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:39.302839 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:39.304727 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:39.305442 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:39.305449 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:39.306033 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:39.306637 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:39.306652 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:39.307577 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:39.307580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:39.308960 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:39.308975 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:39.309511 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:39.309518 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:39.309848 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:39.310595 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:39.310622 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:39.312914 (+ 2292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:39.314753 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:39.314763 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:39.314781 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:39.315163 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:39.315435 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:39.315441 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:39.315446 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:39.315649 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:39.316045 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:39.316055 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:39.316438 (+ 383us) client_negotiation.cc:769] Sending connection context
0806 00:48:39.316660 (+ 222us) client_negotiation.cc:240] Negotiation successful
0806 00:48:39.317098 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":186,"threads_started":1}
I20260806 00:48:39.319644 16487 heartbeater.cc:347] Connected to a master server at 127.15.147.190:35027
I20260806 00:48:39.319893 16487 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.147.131@KRBTEST.COM; defaulting to no policy
I20260806 00:48:39.320467 16487 heartbeater.cc:511] Master 127.15.147.190:35027 requested a full tablet report, sending...
I20260806 00:48:39.321538 16027 ts_manager.cc:194] Registered new tserver with Master: 2cc3729bbc584524a37dfd5d6fdb1f4b (127.15.147.130:38863)
I20260806 00:48:39.322103 16027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:54305
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
I20260806 00:48:39.341056 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:35027
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:35305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:39.490722 16495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:39.491012 16495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:39.491060 16495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:39.495265 16495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:39.495321 16495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:39.495412 16495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:48:39.500820 16495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:35305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:35027
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:39.501912 16495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:39.503196 16495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:39.514487 16507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:39.514535 16505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:39.514557 16504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:39.514876 16495 server_base.cc:1034] running on GCE node
I20260806 00:48:39.515249 16495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:39.515838 16495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:39.517006 16495 hybrid_clock.cc:648] HybridClock initialized: now 1785977319516992 us; error 26 us; skew 500 ppm
Aug 06 00:48:39 dist-test-slave-45dx krb5kdc[15959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977319, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:39.520202 16495 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:48:39.521071 16495 webserver.cc:460] Webserver started at http://127.15.147.131:40587/ using document root <none> and password file <none>
I20260806 00:48:39.521633 16495 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:39.521725 16495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:39.521982 16495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:39.524179 16495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "bc615441008540fd9a06c38e2eed7b46"
format_stamp: "Formatted at 2026-08-06 00:48:39 on dist-test-slave-45dx"
server_key: "712b739f73b9801ad043ccc471e3653e"
server_key_iv: "62024e7de3b8f8352642493b185fcef2"
server_key_version: "encryptionkey@0"
I20260806 00:48:39.524817 16495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "bc615441008540fd9a06c38e2eed7b46"
format_stamp: "Formatted at 2026-08-06 00:48:39 on dist-test-slave-45dx"
server_key: "712b739f73b9801ad043ccc471e3653e"
server_key_iv: "62024e7de3b8f8352642493b185fcef2"
server_key_version: "encryptionkey@0"
I20260806 00:48:39.528705 16495 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:48:39.531263 16517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:39.532581 16495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:48:39.532728 16495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "bc615441008540fd9a06c38e2eed7b46"
format_stamp: "Formatted at 2026-08-06 00:48:39 on dist-test-slave-45dx"
server_key: "712b739f73b9801ad043ccc471e3653e"
server_key_iv: "62024e7de3b8f8352642493b185fcef2"
server_key_version: "encryptionkey@0"
I20260806 00:48:39.532891 16495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:39.547644 16495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:39.550772 16495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:39.551051 16495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:39.551844 16495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:48:39.552897 16495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:48:39.553052 16495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:39.553174 16495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:48:39.553222 16495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:39.571216 16495 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:43359
I20260806 00:48:39.571244 16677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:43359 every 8 connection(s)
I20260806 00:48:39.572160 16495 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:48:39.578890 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 16495
I20260806 00:48:39.578994 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:48:39.579265 15950 external_mini_cluster.cc:1428] Setting key 5b0159b55993aa30fa69e6ee5bc94f14
Aug 06 00:48:39 dist-test-slave-45dx krb5kdc[15959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977319, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:39.587080 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:39.574118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:52569 (local address 127.15.147.190:35027)
0806 00:48:39.574264 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:39.574268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:39.575203 (+ 935us) server_negotiation.cc:407] Connection header received
0806 00:48:39.576617 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:39.576620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:39.576666 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:39.576741 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:39.578051 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:39.578568 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:39.580061 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:39.580395 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:39.580509 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:39.583574 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:39.583577 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:39.583579 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:39.583603 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:39.585163 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:39.585669 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:39.585672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:39.585673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:39.585751 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:39.586046 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:39.586048 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:39.586049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:39.586217 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:39.586327 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:39.586826 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 00:48:39.586932 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:48:39.587956 16684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:39.574489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35027 (local address 127.15.147.131:52569)
0806 00:48:39.575041 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:39.575086 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:39.576423 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:39.576887 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:39.576909 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:39.577314 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:39.577918 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:39.577934 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:39.578674 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:39.578676 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:39.579955 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:39.579965 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:39.580555 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:39.580559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:39.580800 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:39.581526 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:39.581547 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:39.583326 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:39.585266 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:39.585271 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:39.585281 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:39.585555 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:39.585831 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:39.585833 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:39.585834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:39.585967 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:39.586365 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:39.586370 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:39.586601 (+ 231us) client_negotiation.cc:769] Sending connection context
0806 00:48:39.586758 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 00:48:39.587005 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":169,"threads_started":1}
I20260806 00:48:39.589244 16680 heartbeater.cc:347] Connected to a master server at 127.15.147.190:35027
I20260806 00:48:39.589501 16680 heartbeater.cc:464] Registering TS with master...
I20260806 00:48:39.589988 16680 heartbeater.cc:511] Master 127.15.147.190:35027 requested a full tablet report, sending...
I20260806 00:48:39.591038 16027 ts_manager.cc:194] Registered new tserver with Master: bc615441008540fd9a06c38e2eed7b46 (127.15.147.131:43359)
I20260806 00:48:39.591569 16027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:52569
I20260806 00:48:39.594096 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 00:48:39 dist-test-slave-45dx krb5kdc[15959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977319, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 00:48:39 dist-test-slave-45dx krb5kdc[15959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977319, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.147.129@KRBTEST.COM
I20260806 00:48:39.623715 16689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:39.614058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58350 (local address 127.15.147.129:34995)
0806 00:48:39.614409 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:39.614416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:39.614443 (+ 27us) server_negotiation.cc:407] Connection header received
0806 00:48:39.614558 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:39.614564 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:39.614756 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:39.614881 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:39.615608 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:39.616446 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:39.617301 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:39.617665 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:39.617747 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:39.620354 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:39.620378 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:39.620392 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:39.620418 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:39.622138 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:39.622539 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:39.622544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:39.622545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:39.622625 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:39.622857 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:39.622860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:39.622861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:39.623088 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:39.623244 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:39.623409 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 00:48:39.623535 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":105,"threads_started":1}
W20260806 00:48:39.624923 16175 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58350
Aug 06 00:48:39 dist-test-slave-45dx krb5kdc[15959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 00:48:39 dist-test-slave-45dx krb5kdc[15959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.129@KRBTEST.COM
I20260806 00:48:39.654695 16689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:39.646668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58354 (local address 127.15.147.129:34995)
0806 00:48:39.646825 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:39.646830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:39.646984 (+ 154us) server_negotiation.cc:407] Connection header received
0806 00:48:39.647099 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:39.647102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:39.647144 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:39.647216 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:39.647867 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:39.648421 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:39.649190 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:39.649412 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:39.649487 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:39.651888 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:39.651891 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:39.651894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:39.651914 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:39.653222 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:39.653594 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:39.653597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:39.653598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:39.653673 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:39.653898 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:39.653900 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:39.653901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:39.654051 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:39.654154 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:39.654358 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 00:48:39.654454 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:48:39.655804 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 16111
I20260806 00:48:39.668532 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 16303
I20260806 00:48:39.680773 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 16495
I20260806 00:48:39.689847 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 15979
2026-08-06T00:48:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2651 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:39 dist-test-slave-45dx krb5kdc[16700](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:39 dist-test-slave-45dx krb5kdc[16700](info): set up 2 sockets
Aug 06 00:48:39 dist-test-slave-45dx krb5kdc[16700](info): commencing operation
krb5kdc: starting...
W20260806 00:48:40.944600 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:48:41 dist-test-slave-45dx krb5kdc[16700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:48:41.071326 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:40281
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:35457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:40281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:41.180863 16717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:41.181166 16717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:41.181244 16717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:41.185209 16717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:41.185308 16717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:41.185344 16717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:41.185384 16717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:41.185415 16717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:41.190393 16717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:35457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:40281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:40281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:41.191359 16717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:41.192293 16717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:41.200688 16725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:41.200670 16723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:41.200664 16727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:41.200860 16717 server_base.cc:1034] running on GCE node
I20260806 00:48:41.201463 16717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:41.202462 16717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:41.203629 16717 hybrid_clock.cc:648] HybridClock initialized: now 1785977321203611 us; error 30 us; skew 500 ppm
Aug 06 00:48:41 dist-test-slave-45dx krb5kdc[16700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977321, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:41.206295 16717 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:48:41.207157 16717 webserver.cc:460] Webserver started at http://127.15.147.190:40173/ using document root <none> and password file <none>
I20260806 00:48:41.207670 16717 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:41.207762 16717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:41.207983 16717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:41.209923 16717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "102f430f326b464ab103cb90b9421501"
format_stamp: "Formatted at 2026-08-06 00:48:41 on dist-test-slave-45dx"
server_key: "2221d39a89401a8a214c835879255eeb"
server_key_iv: "729139cb36a237431584ed7130dbb44b"
server_key_version: "encryptionkey@0"
I20260806 00:48:41.210482 16717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "102f430f326b464ab103cb90b9421501"
format_stamp: "Formatted at 2026-08-06 00:48:41 on dist-test-slave-45dx"
server_key: "2221d39a89401a8a214c835879255eeb"
server_key_iv: "729139cb36a237431584ed7130dbb44b"
server_key_version: "encryptionkey@0"
I20260806 00:48:41.214036 16717 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260806 00:48:41.216290 16740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:41.217530 16717 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:48:41.217676 16717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "102f430f326b464ab103cb90b9421501"
format_stamp: "Formatted at 2026-08-06 00:48:41 on dist-test-slave-45dx"
server_key: "2221d39a89401a8a214c835879255eeb"
server_key_iv: "729139cb36a237431584ed7130dbb44b"
server_key_version: "encryptionkey@0"
I20260806 00:48:41.217788 16717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:41.231530 16717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:41.234196 16717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:41.234411 16717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:41.241720 16717 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:40281
I20260806 00:48:41.241736 16813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:40281 every 8 connection(s)
I20260806 00:48:41.242705 16717 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:41.245533 16814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:41.247615 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 16717
I20260806 00:48:41.247740 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:48:41.247996 15950 external_mini_cluster.cc:1428] Setting key 080bf9b0a36a30a00b66a972530f74c1
I20260806 00:48:41.251454 16814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501: Bootstrap starting.
I20260806 00:48:41.253680 16814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:41.254446 16814 log.cc:824] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:41.256865 16814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501: No bootstrap required, opened a new log
Aug 06 00:48:41 dist-test-slave-45dx krb5kdc[16700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:41.260987 16814 raft_consensus.cc:348] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "102f430f326b464ab103cb90b9421501" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40281 } }
I20260806 00:48:41.261336 16814 raft_consensus.cc:374] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:41.261426 16814 raft_consensus.cc:729] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 102f430f326b464ab103cb90b9421501, State: Initialized, Role: FOLLOWER
I20260806 00:48:41.261972 16814 consensus_queue.cc:260] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "102f430f326b464ab103cb90b9421501" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40281 } }
I20260806 00:48:41.262113 16814 raft_consensus.cc:388] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:41.262198 16814 raft_consensus.cc:482] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:41.262295 16814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:41.262938 16818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:41.249496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54696 (local address 127.15.147.190:40281)
0806 00:48:41.249908 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:41.249922 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:41.249955 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:48:41.250587 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:41.250603 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:41.250916 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:41.251281 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:41.252126 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:41.253159 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:41.254092 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:41.254604 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:41.254790 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:41.257745 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:41.257751 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:41.257764 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:41.257790 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:41.259594 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:41.260164 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:41.260169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:41.260174 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:41.260293 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:41.260654 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:41.260656 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:41.260657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:41.261176 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:41.261365 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:41.261693 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 00:48:41.262051 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":125,"threads_started":1}
I20260806 00:48:41.263301 16814 raft_consensus.cc:504] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "102f430f326b464ab103cb90b9421501" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40281 } }
I20260806 00:48:41.263657 16814 leader_election.cc:302] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 102f430f326b464ab103cb90b9421501; no voters:
I20260806 00:48:41.263971 16814 leader_election.cc:288] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:41.264081 16820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:41.264287 16820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501 [term 1 LEADER]: Becoming Leader. State: Replica: 102f430f326b464ab103cb90b9421501, State: Running, Role: LEADER
I20260806 00:48:41.264603 16820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "102f430f326b464ab103cb90b9421501" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40281 } }
I20260806 00:48:41.265287 16814 sys_catalog.cc:564] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:41.266254 16821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "102f430f326b464ab103cb90b9421501" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "102f430f326b464ab103cb90b9421501" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40281 } } }
I20260806 00:48:41.266377 16821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:41.266654 16822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 102f430f326b464ab103cb90b9421501. Latest consensus state: current_term: 1 leader_uuid: "102f430f326b464ab103cb90b9421501" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "102f430f326b464ab103cb90b9421501" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40281 } } }
I20260806 00:48:41.266783 16822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:41.267082 16834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:41.270208 16834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:41.276480 16834 catalog_manager.cc:1347] Generated new cluster ID: e543fa3cd0c14d57a11c1108034bf11e
I20260806 00:48:41.276551 16834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:41.295272 16834 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:41.296491 16834 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:41.303987 16834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 102f430f326b464ab103cb90b9421501: Generated new TSK 0
I20260806 00:48:41.304728 16834 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
I20260806 00:48:41.343475 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:40281
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:35457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:41.455430 16853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:41.455775 16853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:41.455849 16853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:41.459540 16853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:41.459616 16853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:41.459698 16853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:48:41.464704 16853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:35457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:40281
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:41.465798 16853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:41.467151 16853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:41.477176 16858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:41.477196 16865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:41.477465 16853 server_base.cc:1034] running on GCE node
W20260806 00:48:41.477638 16860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:41.478144 16853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:41.478780 16853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:41.479977 16853 hybrid_clock.cc:648] HybridClock initialized: now 1785977321479968 us; error 41 us; skew 500 ppm
Aug 06 00:48:41 dist-test-slave-45dx krb5kdc[16700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977321, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:41.482904 16853 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:48:41.483767 16853 webserver.cc:460] Webserver started at http://127.15.147.129:42747/ using document root <none> and password file <none>
I20260806 00:48:41.484400 16853 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:41.484498 16853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:41.484745 16853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:41.486874 16853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "64a50275ad064a6c90a011843e281814"
format_stamp: "Formatted at 2026-08-06 00:48:41 on dist-test-slave-45dx"
server_key: "7aca22a0be265a7d89913bfca0be0373"
server_key_iv: "9b00baee60d26a7e13a213c77533e6da"
server_key_version: "encryptionkey@0"
I20260806 00:48:41.487488 16853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "64a50275ad064a6c90a011843e281814"
format_stamp: "Formatted at 2026-08-06 00:48:41 on dist-test-slave-45dx"
server_key: "7aca22a0be265a7d89913bfca0be0373"
server_key_iv: "9b00baee60d26a7e13a213c77533e6da"
server_key_version: "encryptionkey@0"
I20260806 00:48:41.491430 16853 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 00:48:41.493973 16876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:41.495208 16853 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:48:41.495386 16853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "64a50275ad064a6c90a011843e281814"
format_stamp: "Formatted at 2026-08-06 00:48:41 on dist-test-slave-45dx"
server_key: "7aca22a0be265a7d89913bfca0be0373"
server_key_iv: "9b00baee60d26a7e13a213c77533e6da"
server_key_version: "encryptionkey@0"
I20260806 00:48:41.495507 16853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:41.511291 16853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:41.514787 16853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:41.515002 16853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:41.515616 16853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:48:41.516535 16853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:48:41.516605 16853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:41.516678 16853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:48:41.516728 16853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:41.525751 16853 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:46419
I20260806 00:48:41.525794 17040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:46419 every 8 connection(s)
I20260806 00:48:41.526611 16853 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:48:41.530755 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 16853
I20260806 00:48:41.530864 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:48:41.531131 15950 external_mini_cluster.cc:1428] Setting key 50e0088a940c7057a3bb11d68a942959
Aug 06 00:48:41 dist-test-slave-45dx krb5kdc[16700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977321, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:41.540546 16818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:41.528323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:52767 (local address 127.15.147.190:40281)
0806 00:48:41.528458 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:41.528462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:41.529282 (+ 820us) server_negotiation.cc:407] Connection header received
0806 00:48:41.530106 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:41.530108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:41.530171 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:41.530244 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:41.531475 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:41.532002 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:41.533221 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:41.533463 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:41.533572 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:41.536375 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:41.536382 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:41.536388 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:41.536421 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:41.538267 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:41.538803 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:41.538810 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:41.538815 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:41.538925 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:41.539303 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:41.539308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:41.539312 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:41.539599 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:41.539720 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:41.540184 (+ 464us) server_negotiation.cc:299] Negotiation successful
0806 00:48:41.540342 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:48:41.541368 17043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:41.528710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40281 (local address 127.15.147.129:52767)
0806 00:48:41.529128 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:41.529167 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:41.529912 (+ 745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:41.530423 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:41.530429 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:41.530746 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:41.531327 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:41.531341 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:41.532139 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:41.532142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:41.533081 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:41.533091 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:41.533635 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:41.533639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:41.533794 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:41.534479 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:41.534505 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:41.536172 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:41.538409 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:41.538414 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:41.538425 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:41.538678 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:41.539047 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:41.539050 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:41.539051 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:41.539202 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:41.539713 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:41.539718 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:41.539938 (+ 220us) client_negotiation.cc:769] Sending connection context
0806 00:48:41.540092 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 00:48:41.540507 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for kudu/127.15.147.130@KRBTEST.COM; defaulting to no policy
I20260806 00:48:41.542636 17041 heartbeater.cc:347] Connected to a master server at 127.15.147.190:40281
I20260806 00:48:41.542897 17041 heartbeater.cc:464] Registering TS with master...
I20260806 00:48:41.543385 17041 heartbeater.cc:511] Master 127.15.147.190:40281 requested a full tablet report, sending...
I20260806 00:48:41.544894 16762 ts_manager.cc:194] Registered new tserver with Master: 64a50275ad064a6c90a011843e281814 (127.15.147.129:46419)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.130@KRBTEST.COM" created.
I20260806 00:48:41.546283 16762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:52767
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
I20260806 00:48:41.562716 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:40281
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:35457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:41.677254 17050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:41.677539 17050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:41.677611 17050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:41.681566 17050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:41.681667 17050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:41.681782 17050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:48:41.687058 17050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:35457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:40281
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:41.688095 17050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:41.689111 17050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:41.698711 17056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:41.698741 17060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:41.698745 17057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:41.698987 17050 server_base.cc:1034] running on GCE node
I20260806 00:48:41.699414 17050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:41.700053 17050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:41.701236 17050 hybrid_clock.cc:648] HybridClock initialized: now 1785977321701210 us; error 48 us; skew 500 ppm
Aug 06 00:48:41 dist-test-slave-45dx krb5kdc[16700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977321, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:41.703949 17050 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:48:41.704892 17050 webserver.cc:460] Webserver started at http://127.15.147.130:35855/ using document root <none> and password file <none>
I20260806 00:48:41.705459 17050 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:41.705550 17050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:41.705775 17050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:41.707707 17050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "dc9eb9a04439423bb3023f6ce60394dd"
format_stamp: "Formatted at 2026-08-06 00:48:41 on dist-test-slave-45dx"
server_key: "c0f03114ba746a6268a64e31b9de94c1"
server_key_iv: "1cc85b22a76e6a9c0cfb781bcbb8ef8e"
server_key_version: "encryptionkey@0"
I20260806 00:48:41.708274 17050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "dc9eb9a04439423bb3023f6ce60394dd"
format_stamp: "Formatted at 2026-08-06 00:48:41 on dist-test-slave-45dx"
server_key: "c0f03114ba746a6268a64e31b9de94c1"
server_key_iv: "1cc85b22a76e6a9c0cfb781bcbb8ef8e"
server_key_version: "encryptionkey@0"
I20260806 00:48:41.711889 17050 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:48:41.714268 17069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:41.715412 17050 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:48:41.715545 17050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "dc9eb9a04439423bb3023f6ce60394dd"
format_stamp: "Formatted at 2026-08-06 00:48:41 on dist-test-slave-45dx"
server_key: "c0f03114ba746a6268a64e31b9de94c1"
server_key_iv: "1cc85b22a76e6a9c0cfb781bcbb8ef8e"
server_key_version: "encryptionkey@0"
I20260806 00:48:41.715653 17050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:41.732733 17050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:41.735644 17050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:41.735859 17050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:41.736577 17050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:48:41.737426 17050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:48:41.737500 17050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:41.737573 17050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:48:41.737617 17050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:41.747670 17050 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:41395
I20260806 00:48:41.747694 17236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:41395 every 8 connection(s)
I20260806 00:48:41.748672 17050 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:48:41.749369 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 17050
I20260806 00:48:41.749487 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:48:41.749770 15950 external_mini_cluster.cc:1428] Setting key eada1b3e905e4048428c641b93f4beeb
Aug 06 00:48:41 dist-test-slave-45dx krb5kdc[16700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977321, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.147.131@KRBTEST.COM; defaulting to no policy
I20260806 00:48:41.764595 16818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:41.750643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:48813 (local address 127.15.147.190:40281)
0806 00:48:41.750791 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:41.750796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:41.751913 (+ 1117us) server_negotiation.cc:407] Connection header received
0806 00:48:41.753269 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:41.753271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:41.753324 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:41.753395 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:41.754739 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:41.755484 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:41.757529 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:41.757747 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:41.757824 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:41.760836 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:41.760840 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:41.760842 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:41.760864 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:41.762365 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:41.762937 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:41.762941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:41.762942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:41.763021 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:41.763418 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:41.763421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:41.763423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:41.763634 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:41.763735 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:41.764262 (+ 527us) server_negotiation.cc:299] Negotiation successful
0806 00:48:41.764426 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.131@KRBTEST.COM" created.
I20260806 00:48:41.765684 17239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:41.751088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40281 (local address 127.15.147.130:48813)
0806 00:48:41.751699 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:41.751776 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:41.753042 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:41.753630 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:41.753642 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:41.754084 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:41.754556 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:41.754569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:41.755617 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:41.755623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:41.757376 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:41.757390 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:41.757911 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:41.757919 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:41.758150 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:41.758842 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:41.758868 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:41.760661 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:41.762485 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:41.762492 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:41.762506 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:41.762794 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:41.763117 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:41.763120 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:41.763122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:41.763280 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:41.763736 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:41.763743 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:41.764013 (+ 270us) client_negotiation.cc:769] Sending connection context
0806 00:48:41.764202 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 00:48:41.764620 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":139,"threads_started":1}
I20260806 00:48:41.767014 17237 heartbeater.cc:347] Connected to a master server at 127.15.147.190:40281
I20260806 00:48:41.767293 17237 heartbeater.cc:464] Registering TS with master...
I20260806 00:48:41.767848 17237 heartbeater.cc:511] Master 127.15.147.190:40281 requested a full tablet report, sending...
I20260806 00:48:41.769001 16762 ts_manager.cc:194] Registered new tserver with Master: dc9eb9a04439423bb3023f6ce60394dd (127.15.147.130:41395)
I20260806 00:48:41.769865 16762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:48813
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
I20260806 00:48:41.782632 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:40281
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:35457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:41.897228 17244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:41.897477 17244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:41.897550 17244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:41.901672 17244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:41.901750 17244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:41.901904 17244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:48:41.906996 17244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:35457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:40281
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:41.907975 17244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:41.909041 17244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:41.918426 17254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:41.918442 17251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:41.918519 17244 server_base.cc:1034] running on GCE node
W20260806 00:48:41.918751 17252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:41.919202 17244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:41.919786 17244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:41.920993 17244 hybrid_clock.cc:648] HybridClock initialized: now 1785977321920988 us; error 41 us; skew 500 ppm
Aug 06 00:48:41 dist-test-slave-45dx krb5kdc[16700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977321, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:41.923619 17244 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:48:41.924592 17244 webserver.cc:460] Webserver started at http://127.15.147.131:39999/ using document root <none> and password file <none>
I20260806 00:48:41.925143 17244 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:41.925233 17244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:41.925468 17244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:41.927487 17244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "d2239a98b20f4ecab5ca10d8f082013e"
format_stamp: "Formatted at 2026-08-06 00:48:41 on dist-test-slave-45dx"
server_key: "47839e0e4d8e64dcfd8f27f74559c954"
server_key_iv: "9c81ffa4a8999c27e52bc49d6b207efc"
server_key_version: "encryptionkey@0"
I20260806 00:48:41.928064 17244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "d2239a98b20f4ecab5ca10d8f082013e"
format_stamp: "Formatted at 2026-08-06 00:48:41 on dist-test-slave-45dx"
server_key: "47839e0e4d8e64dcfd8f27f74559c954"
server_key_iv: "9c81ffa4a8999c27e52bc49d6b207efc"
server_key_version: "encryptionkey@0"
I20260806 00:48:41.931636 17244 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:48:41.934113 17266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:41.935266 17244 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:48:41.935427 17244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "d2239a98b20f4ecab5ca10d8f082013e"
format_stamp: "Formatted at 2026-08-06 00:48:41 on dist-test-slave-45dx"
server_key: "47839e0e4d8e64dcfd8f27f74559c954"
server_key_iv: "9c81ffa4a8999c27e52bc49d6b207efc"
server_key_version: "encryptionkey@0"
I20260806 00:48:41.935565 17244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:41.958837 17244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:41.962247 17244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:41.962484 17244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:41.963088 17244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:48:41.963912 17244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:48:41.963984 17244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:41.964047 17244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:48:41.964098 17244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:41.975841 17244 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:33419
I20260806 00:48:41.975879 17438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:33419 every 8 connection(s)
I20260806 00:48:41.976893 17244 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:48:41.979578 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 17244
I20260806 00:48:41.979717 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:48:41.980003 15950 external_mini_cluster.cc:1428] Setting key 6da9b42467a44ef6d7a50ddd6f73e37e
Aug 06 00:48:41 dist-test-slave-45dx krb5kdc[16700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977321, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:41.991413 16818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:41.978895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:58185 (local address 127.15.147.190:40281)
0806 00:48:41.979035 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:41.979039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:41.980129 (+ 1090us) server_negotiation.cc:407] Connection header received
0806 00:48:41.981316 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:41.981320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:41.981387 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:41.981501 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:41.982802 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:41.983328 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:41.984702 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:41.984944 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:41.985058 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:41.987983 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:41.987986 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:41.987988 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:41.988012 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:41.989328 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:41.989847 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:41.989851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:41.989857 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:41.989934 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:41.990276 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:41.990280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:41.990282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:41.990473 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:41.990585 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:41.991090 (+ 505us) server_negotiation.cc:299] Negotiation successful
0806 00:48:41.991206 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:48:41.992235 17443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:41.979191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40281 (local address 127.15.147.131:58185)
0806 00:48:41.979926 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:41.980000 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:41.981106 (+ 1106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:41.981684 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:41.981691 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:41.982060 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:41.982648 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:41.982664 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:41.983468 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:41.983478 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:41.984569 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:41.984579 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:41.985102 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:41.985106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:41.985336 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:41.986007 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:41.986035 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:41.987785 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:41.989460 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:41.989466 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:41.989479 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:41.989734 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:41.990029 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:41.990033 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:41.990035 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:41.990178 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:41.990585 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:41.990591 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:41.990839 (+ 248us) client_negotiation.cc:769] Sending connection context
0806 00:48:41.991033 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 00:48:41.991383 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":271,"threads_started":1}
I20260806 00:48:41.993510 17440 heartbeater.cc:347] Connected to a master server at 127.15.147.190:40281
I20260806 00:48:41.993783 17440 heartbeater.cc:464] Registering TS with master...
I20260806 00:48:41.994304 17440 heartbeater.cc:511] Master 127.15.147.190:40281 requested a full tablet report, sending...
I20260806 00:48:41.995265 16761 ts_manager.cc:194] Registered new tserver with Master: d2239a98b20f4ecab5ca10d8f082013e (127.15.147.131:33419)
I20260806 00:48:41.995838 16761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:58185
I20260806 00:48:42.005636 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:48:42.016795 16818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:42.009777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54698 (local address 127.15.147.190:40281)
0806 00:48:42.009904 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:42.009907 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:42.010224 (+ 317us) server_negotiation.cc:407] Connection header received
0806 00:48:42.010320 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:42.010323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:42.010358 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:42.010427 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:42.011169 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.011648 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.012436 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.012641 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.012717 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:42.014027 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:42.014030 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:42.014032 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:42.014053 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:42.015284 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:42.015649 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:42.015651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:42.015652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:42.015723 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:42.015948 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:42.015950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:42.015951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:42.016155 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:42.016271 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:42.016506 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 00:48:42.016600 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:48:42.022688 16761 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:48:42.026085 16761 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:48:42.043495 17455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:42.036068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:41395 (local address 127.0.0.1:41976)
0806 00:48:42.036609 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:42.036667 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:42.036830 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:42.037495 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:42.037502 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:42.037526 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:42.037809 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:42.037818 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.040104 (+ 2286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.040107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:42.041633 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:42.041645 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.042849 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.042853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:42.042993 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:42.043036 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 00:48:42.043183 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 00:48:42.043355 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":57,"thread_start_us":79,"threads_started":1}
I20260806 00:48:42.043571 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:42.036216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.15.147.130:41395)
0806 00:48:42.036953 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:42.036994 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:42.037019 (+ 25us) server_negotiation.cc:407] Connection header received
0806 00:48:42.037099 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:42.037106 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:42.037317 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:42.037986 (+ 669us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:42.038352 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.039818 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.041815 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.042703 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.042850 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:42.043050 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:42.043217 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 00:48:42.043329 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":321,"threads_started":1}
I20260806 00:48:42.043565 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:42.036147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:46419 (local address 127.0.0.1:54614)
0806 00:48:42.037100 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:42.037112 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:42.037189 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:42.038596 (+ 1407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:42.038599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:42.038630 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:42.038790 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:42.038797 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.040789 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.040792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:42.041691 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:42.041702 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.042925 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.042929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:42.043055 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:42.043083 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 00:48:42.043164 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 00:48:42.043370 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":3,"spinlock_wait_cycles":768,"thread_start_us":238,"threads_started":1}
I20260806 00:48:42.044173 17459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:42.036907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:33419 (local address 127.0.0.1:44786)
0806 00:48:42.037539 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:42.037552 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:42.037624 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:42.038557 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:42.038559 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:42.038578 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:42.038823 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:42.038839 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.040409 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.040412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:42.041919 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:42.041928 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.043527 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.043531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:42.043693 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:42.043716 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:48:42.043772 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 00:48:42.044048 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"mutex_wait_us":220,"thread_start_us":182,"threads_started":1}
I20260806 00:48:42.044858 17460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:42.036983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54614 (local address 127.15.147.129:46419)
0806 00:48:42.037926 (+ 943us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:42.037943 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:42.037964 (+ 21us) server_negotiation.cc:407] Connection header received
0806 00:48:42.038055 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:42.038060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:42.038365 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:42.038952 (+ 587us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:42.039157 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.040488 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.041844 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.042774 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.044416 (+ 1642us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:42.044541 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:42.044587 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 00:48:42.044670 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":820,"thread_start_us":133,"threads_started":1}
I20260806 00:48:42.045065 17461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:42.037445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44786 (local address 127.15.147.131:33419)
0806 00:48:42.038096 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:42.038162 (+ 66us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:42.038188 (+ 26us) server_negotiation.cc:407] Connection header received
0806 00:48:42.038267 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:42.038271 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:42.038418 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:42.038574 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:42.039037 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.039940 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.042252 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.043389 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.044595 (+ 1206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:42.044692 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:42.044733 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 00:48:42.044830 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":185,"threads_started":1}
I20260806 00:48:42.047391 17140 tablet_service.cc:1467] Processing CreateTablet for tablet 8a83a8078e424e589c4f63602109595a (DEFAULT_TABLE table=test-table [id=9cf636fa15b24ef69175482c3b881ebe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:42.048058 17334 tablet_service.cc:1467] Processing CreateTablet for tablet 8a83a8078e424e589c4f63602109595a (DEFAULT_TABLE table=test-table [id=9cf636fa15b24ef69175482c3b881ebe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:42.048056 16946 tablet_service.cc:1467] Processing CreateTablet for tablet 8a83a8078e424e589c4f63602109595a (DEFAULT_TABLE table=test-table [id=9cf636fa15b24ef69175482c3b881ebe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:42.049053 17140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a83a8078e424e589c4f63602109595a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:42.049219 16946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a83a8078e424e589c4f63602109595a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:42.049301 17334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a83a8078e424e589c4f63602109595a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:42.057468 17462 tablet_bootstrap.cc:492] T 8a83a8078e424e589c4f63602109595a P dc9eb9a04439423bb3023f6ce60394dd: Bootstrap starting.
I20260806 00:48:42.058811 17464 tablet_bootstrap.cc:492] T 8a83a8078e424e589c4f63602109595a P 64a50275ad064a6c90a011843e281814: Bootstrap starting.
I20260806 00:48:42.059299 17463 tablet_bootstrap.cc:492] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e: Bootstrap starting.
I20260806 00:48:42.061960 17463 tablet_bootstrap.cc:654] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:42.062170 17464 tablet_bootstrap.cc:654] T 8a83a8078e424e589c4f63602109595a P 64a50275ad064a6c90a011843e281814: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:42.062858 17462 tablet_bootstrap.cc:654] T 8a83a8078e424e589c4f63602109595a P dc9eb9a04439423bb3023f6ce60394dd: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:42.063812 17463 log.cc:824] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:42.064069 17464 log.cc:824] T 8a83a8078e424e589c4f63602109595a P 64a50275ad064a6c90a011843e281814: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:42.064497 17462 log.cc:824] T 8a83a8078e424e589c4f63602109595a P dc9eb9a04439423bb3023f6ce60394dd: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:42.066478 17463 tablet_bootstrap.cc:492] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e: No bootstrap required, opened a new log
I20260806 00:48:42.066684 17463 ts_tablet_manager.cc:1397] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260806 00:48:42.066903 17464 tablet_bootstrap.cc:492] T 8a83a8078e424e589c4f63602109595a P 64a50275ad064a6c90a011843e281814: No bootstrap required, opened a new log
I20260806 00:48:42.067062 17462 tablet_bootstrap.cc:492] T 8a83a8078e424e589c4f63602109595a P dc9eb9a04439423bb3023f6ce60394dd: No bootstrap required, opened a new log
I20260806 00:48:42.067191 17464 ts_tablet_manager.cc:1397] T 8a83a8078e424e589c4f63602109595a P 64a50275ad064a6c90a011843e281814: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260806 00:48:42.067272 17462 ts_tablet_manager.cc:1397] T 8a83a8078e424e589c4f63602109595a P dc9eb9a04439423bb3023f6ce60394dd: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260806 00:48:42.071904 17462 raft_consensus.cc:348] T 8a83a8078e424e589c4f63602109595a P dc9eb9a04439423bb3023f6ce60394dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9eb9a04439423bb3023f6ce60394dd" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 41395 } } peers { permanent_uuid: "64a50275ad064a6c90a011843e281814" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46419 } } peers { permanent_uuid: "d2239a98b20f4ecab5ca10d8f082013e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 33419 } }
I20260806 00:48:42.072230 17462 raft_consensus.cc:374] T 8a83a8078e424e589c4f63602109595a P dc9eb9a04439423bb3023f6ce60394dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:42.072326 17462 raft_consensus.cc:729] T 8a83a8078e424e589c4f63602109595a P dc9eb9a04439423bb3023f6ce60394dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9eb9a04439423bb3023f6ce60394dd, State: Initialized, Role: FOLLOWER
I20260806 00:48:42.072808 17462 consensus_queue.cc:260] T 8a83a8078e424e589c4f63602109595a P dc9eb9a04439423bb3023f6ce60394dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9eb9a04439423bb3023f6ce60394dd" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 41395 } } peers { permanent_uuid: "64a50275ad064a6c90a011843e281814" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46419 } } peers { permanent_uuid: "d2239a98b20f4ecab5ca10d8f082013e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 33419 } }
I20260806 00:48:42.073868 17237 heartbeater.cc:503] Master 127.15.147.190:40281 was elected leader, sending a full tablet report...
I20260806 00:48:42.073988 17462 ts_tablet_manager.cc:1428] T 8a83a8078e424e589c4f63602109595a P dc9eb9a04439423bb3023f6ce60394dd: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260806 00:48:42.073733 17463 raft_consensus.cc:348] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9eb9a04439423bb3023f6ce60394dd" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 41395 } } peers { permanent_uuid: "64a50275ad064a6c90a011843e281814" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46419 } } peers { permanent_uuid: "d2239a98b20f4ecab5ca10d8f082013e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 33419 } }
I20260806 00:48:42.074119 17463 raft_consensus.cc:374] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:42.074175 17463 raft_consensus.cc:729] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2239a98b20f4ecab5ca10d8f082013e, State: Initialized, Role: FOLLOWER
I20260806 00:48:42.074633 17463 consensus_queue.cc:260] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9eb9a04439423bb3023f6ce60394dd" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 41395 } } peers { permanent_uuid: "64a50275ad064a6c90a011843e281814" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46419 } } peers { permanent_uuid: "d2239a98b20f4ecab5ca10d8f082013e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 33419 } }
I20260806 00:48:42.074970 17464 raft_consensus.cc:348] T 8a83a8078e424e589c4f63602109595a P 64a50275ad064a6c90a011843e281814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9eb9a04439423bb3023f6ce60394dd" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 41395 } } peers { permanent_uuid: "64a50275ad064a6c90a011843e281814" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46419 } } peers { permanent_uuid: "d2239a98b20f4ecab5ca10d8f082013e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 33419 } }
I20260806 00:48:42.075259 17464 raft_consensus.cc:374] T 8a83a8078e424e589c4f63602109595a P 64a50275ad064a6c90a011843e281814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:42.075344 17464 raft_consensus.cc:729] T 8a83a8078e424e589c4f63602109595a P 64a50275ad064a6c90a011843e281814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64a50275ad064a6c90a011843e281814, State: Initialized, Role: FOLLOWER
I20260806 00:48:42.075613 17463 ts_tablet_manager.cc:1428] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260806 00:48:42.075964 17440 heartbeater.cc:503] Master 127.15.147.190:40281 was elected leader, sending a full tablet report...
I20260806 00:48:42.076082 17464 consensus_queue.cc:260] T 8a83a8078e424e589c4f63602109595a P 64a50275ad064a6c90a011843e281814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9eb9a04439423bb3023f6ce60394dd" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 41395 } } peers { permanent_uuid: "64a50275ad064a6c90a011843e281814" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46419 } } peers { permanent_uuid: "d2239a98b20f4ecab5ca10d8f082013e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 33419 } }
I20260806 00:48:42.076869 17041 heartbeater.cc:503] Master 127.15.147.190:40281 was elected leader, sending a full tablet report...
I20260806 00:48:42.078554 17464 ts_tablet_manager.cc:1428] T 8a83a8078e424e589c4f63602109595a P 64a50275ad064a6c90a011843e281814: Time spent starting tablet: real 0.011s user 0.005s sys 0.006s
W20260806 00:48:42.228214 17441 tablet.cc:2369] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:48:42.250105 17238 tablet.cc:2369] T 8a83a8078e424e589c4f63602109595a P dc9eb9a04439423bb3023f6ce60394dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:48:42.263581 17472 raft_consensus.cc:482] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:48:42.263851 17472 raft_consensus.cc:504] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9eb9a04439423bb3023f6ce60394dd" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 41395 } } peers { permanent_uuid: "64a50275ad064a6c90a011843e281814" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46419 } } peers { permanent_uuid: "d2239a98b20f4ecab5ca10d8f082013e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 33419 } }
I20260806 00:48:42.266824 17472 leader_election.cc:288] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc9eb9a04439423bb3023f6ce60394dd (127.15.147.130:41395), 64a50275ad064a6c90a011843e281814 (127.15.147.129:46419)
I20260806 00:48:42.272131 17443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:42.266253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:46419 (local address 127.15.147.131:60909)
0806 00:48:42.266808 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:42.266837 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:42.266976 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:42.267612 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:42.267615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:42.267633 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:42.268001 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:42.268011 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.269156 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.269161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:42.270452 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:42.270463 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.271597 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.271601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:42.271791 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:42.271817 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:48:42.271890 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 00:48:42.271960 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467}
I20260806 00:48:42.272570 17460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:42.266732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:60909 (local address 127.15.147.129:46419)
0806 00:48:42.267265 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:42.267269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:42.267287 (+ 18us) server_negotiation.cc:407] Connection header received
0806 00:48:42.267345 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:42.267348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:42.267404 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:42.267480 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:42.268171 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.269016 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.270641 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.271369 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.271483 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:42.271536 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:42.272340 (+ 804us) server_negotiation.cc:299] Negotiation successful
0806 00:48:42.272439 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396}
I20260806 00:48:42.272636 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:42.266380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:41213 (local address 127.15.147.130:41395)
0806 00:48:42.266593 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:42.266598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:42.267424 (+ 826us) server_negotiation.cc:407] Connection header received
0806 00:48:42.267603 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:42.267607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:42.267662 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:42.267735 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:42.268517 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.269222 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.271030 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.271644 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.271800 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:42.271847 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:42.272427 (+ 580us) server_negotiation.cc:299] Negotiation successful
0806 00:48:42.272517 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 00:48:42.273046 17474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:42.266292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:41395 (local address 127.15.147.131:41213)
0806 00:48:42.267229 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:42.267244 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:42.267444 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:42.267884 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:42.267887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:42.267908 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:42.268328 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:42.268343 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.269338 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.269343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:42.270883 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:42.270895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.271863 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.271869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:42.272203 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:42.272237 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 00:48:42.272776 (+ 539us) client_negotiation.cc:240] Negotiation successful
0806 00:48:42.272862 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"thread_start_us":222,"threads_started":1}
I20260806 00:48:42.273407 16973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a83a8078e424e589c4f63602109595a" candidate_uuid: "d2239a98b20f4ecab5ca10d8f082013e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64a50275ad064a6c90a011843e281814" is_pre_election: true
I20260806 00:48:42.273828 17166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a83a8078e424e589c4f63602109595a" candidate_uuid: "d2239a98b20f4ecab5ca10d8f082013e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9eb9a04439423bb3023f6ce60394dd" is_pre_election: true
I20260806 00:48:42.273937 16973 raft_consensus.cc:2454] T 8a83a8078e424e589c4f63602109595a P 64a50275ad064a6c90a011843e281814 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2239a98b20f4ecab5ca10d8f082013e in term 0.
I20260806 00:48:42.274170 17166 raft_consensus.cc:2454] T 8a83a8078e424e589c4f63602109595a P dc9eb9a04439423bb3023f6ce60394dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2239a98b20f4ecab5ca10d8f082013e in term 0.
I20260806 00:48:42.274706 17270 leader_election.cc:302] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64a50275ad064a6c90a011843e281814, d2239a98b20f4ecab5ca10d8f082013e; no voters:
I20260806 00:48:42.275013 17472 raft_consensus.cc:2781] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:48:42.275112 17472 raft_consensus.cc:482] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:48:42.275167 17472 raft_consensus.cc:3037] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:42.276605 17472 raft_consensus.cc:504] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9eb9a04439423bb3023f6ce60394dd" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 41395 } } peers { permanent_uuid: "64a50275ad064a6c90a011843e281814" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46419 } } peers { permanent_uuid: "d2239a98b20f4ecab5ca10d8f082013e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 33419 } }
I20260806 00:48:42.276942 17472 leader_election.cc:288] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e [CANDIDATE]: Term 1 election: Requested vote from peers dc9eb9a04439423bb3023f6ce60394dd (127.15.147.130:41395), 64a50275ad064a6c90a011843e281814 (127.15.147.129:46419)
I20260806 00:48:42.277377 17166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a83a8078e424e589c4f63602109595a" candidate_uuid: "d2239a98b20f4ecab5ca10d8f082013e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc9eb9a04439423bb3023f6ce60394dd"
I20260806 00:48:42.277479 16973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a83a8078e424e589c4f63602109595a" candidate_uuid: "d2239a98b20f4ecab5ca10d8f082013e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64a50275ad064a6c90a011843e281814"
I20260806 00:48:42.277540 17166 raft_consensus.cc:3037] T 8a83a8078e424e589c4f63602109595a P dc9eb9a04439423bb3023f6ce60394dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:42.277592 16973 raft_consensus.cc:3037] T 8a83a8078e424e589c4f63602109595a P 64a50275ad064a6c90a011843e281814 [term 0 FOLLOWER]: Advancing to term 1
W20260806 00:48:42.277937 17042 tablet.cc:2369] T 8a83a8078e424e589c4f63602109595a P 64a50275ad064a6c90a011843e281814: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:48:42.279827 17166 raft_consensus.cc:2454] T 8a83a8078e424e589c4f63602109595a P dc9eb9a04439423bb3023f6ce60394dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2239a98b20f4ecab5ca10d8f082013e in term 1.
I20260806 00:48:42.279827 16973 raft_consensus.cc:2454] T 8a83a8078e424e589c4f63602109595a P 64a50275ad064a6c90a011843e281814 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2239a98b20f4ecab5ca10d8f082013e in term 1.
I20260806 00:48:42.280148 17270 leader_election.cc:302] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64a50275ad064a6c90a011843e281814, d2239a98b20f4ecab5ca10d8f082013e; no voters:
I20260806 00:48:42.280385 17472 raft_consensus.cc:2781] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:42.280627 17472 raft_consensus.cc:686] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e [term 1 LEADER]: Becoming Leader. State: Replica: d2239a98b20f4ecab5ca10d8f082013e, State: Running, Role: LEADER
I20260806 00:48:42.280936 17472 consensus_queue.cc:237] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9eb9a04439423bb3023f6ce60394dd" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 41395 } } peers { permanent_uuid: "64a50275ad064a6c90a011843e281814" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46419 } } peers { permanent_uuid: "d2239a98b20f4ecab5ca10d8f082013e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 33419 } }
I20260806 00:48:42.284092 16762 catalog_manager.cc:5495] T 8a83a8078e424e589c4f63602109595a P d2239a98b20f4ecab5ca10d8f082013e reported cstate change: term changed from 0 to 1, leader changed from <none> to d2239a98b20f4ecab5ca10d8f082013e (127.15.147.131). New cstate: current_term: 1 leader_uuid: "d2239a98b20f4ecab5ca10d8f082013e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9eb9a04439423bb3023f6ce60394dd" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 41395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64a50275ad064a6c90a011843e281814" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2239a98b20f4ecab5ca10d8f082013e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 33419 } health_report { overall_health: HEALTHY } } }
Aug 06 00:48:42 dist-test-slave-45dx krb5kdc[16700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.129@KRBTEST.COM
I20260806 00:48:42.345999 17460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:42.332041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55030 (local address 127.15.147.129:46419)
0806 00:48:42.332238 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:42.332243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:42.332469 (+ 226us) server_negotiation.cc:407] Connection header received
0806 00:48:42.332636 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:42.332639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:42.332685 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:42.332770 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:42.333601 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.334113 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.335229 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.335452 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.335564 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:42.341902 (+ 6338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:42.341918 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:42.341934 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:42.341963 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:42.344010 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:42.344588 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:42.344595 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:42.344596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:42.344737 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:42.345079 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:42.345082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:42.345083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:42.345362 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:42.345532 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:42.345744 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 00:48:42.345859 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
Aug 06 00:48:42 dist-test-slave-45dx krb5kdc[16700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977322, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 00:48:42 dist-test-slave-45dx krb5kdc[16700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977322, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.147.129@KRBTEST.COM
I20260806 00:48:42.376739 17460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:42.368541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55042 (local address 127.15.147.129:46419)
0806 00:48:42.368705 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:42.368709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:42.368774 (+ 65us) server_negotiation.cc:407] Connection header received
0806 00:48:42.368889 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:42.368891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:42.368935 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:42.369005 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:42.369747 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.370254 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.371010 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.371234 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.371308 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:42.373781 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:42.373785 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:42.373787 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:42.373809 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:42.375170 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:42.375544 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:42.375547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:42.375548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:42.375624 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:42.375848 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:42.375850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:42.375851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:42.376001 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:42.376160 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:42.376397 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 00:48:42.376529 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 00:48:42 dist-test-slave-45dx krb5kdc[16700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 00:48:42 dist-test-slave-45dx krb5kdc[16700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.129@KRBTEST.COM
I20260806 00:48:42.404862 17460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:42.396380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55054 (local address 127.15.147.129:46419)
0806 00:48:42.396541 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:42.396545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:42.396682 (+ 137us) server_negotiation.cc:407] Connection header received
0806 00:48:42.396846 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:42.396850 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:42.396910 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:42.397010 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:42.397669 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.398160 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.399166 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:42.399385 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:42.399500 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:42.401905 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:42.401908 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:42.401910 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:42.401932 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:42.403140 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:42.403578 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:42.403581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:42.403582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:42.403669 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:42.403941 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:42.403944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:42.403946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:42.404105 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:42.404237 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:42.404565 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 00:48:42.404697 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 00:48:42.406616 16917 tablet_service.cc:2908] Rejecting scan request for tablet 8a83a8078e424e589c4f63602109595a: Uninitialized: clean time has not yet been initialized
I20260806 00:48:42.408990 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 16853
I20260806 00:48:42.423206 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 17050
I20260806 00:48:42.432478 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 17244
I20260806 00:48:42.441390 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 16717
2026-08-06T00:48:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2750 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:42 dist-test-slave-45dx krb5kdc[17496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:42 dist-test-slave-45dx krb5kdc[17496](info): set up 2 sockets
Aug 06 00:48:42 dist-test-slave-45dx krb5kdc[17496](info): commencing operation
krb5kdc: starting...
W20260806 00:48:43.655333 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:48:43 dist-test-slave-45dx krb5kdc[17496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:48:43.782676 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:41995
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:41995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:43.894418 17514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:43.894696 17514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:43.894804 17514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:43.898638 17514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:43.898715 17514 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 00:48:43.898743 17514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:43.898802 17514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:43.898862 17514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:43.898896 17514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:43.905031 17514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:41995
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:41995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:43.906097 17514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:43.907135 17514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:43.915584 17520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:43.915616 17521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:43.915738 17514 server_base.cc:1034] running on GCE node
W20260806 00:48:43.915611 17523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:43.916571 17514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:43.917662 17514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:43.918865 17514 hybrid_clock.cc:648] HybridClock initialized: now 1785977323918831 us; error 44 us; skew 500 ppm
Aug 06 00:48:43 dist-test-slave-45dx krb5kdc[17496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977323, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:43.921723 17514 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:48:43.922703 17514 webserver.cc:460] Webserver started at http://127.15.147.190:36041/ using document root <none> and password file <none>
I20260806 00:48:43.923298 17514 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:43.923398 17514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:43.923645 17514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:43.925875 17514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "85d314529f6e4b13bbedf99781761f65"
format_stamp: "Formatted at 2026-08-06 00:48:43 on dist-test-slave-45dx"
server_key: "c3a4a8d6feffa175f0c32cf5ca95f7b6"
server_key_iv: "191a4a71275b17b3c9b954f5f5bdf6a5"
server_key_version: "encryptionkey@0"
I20260806 00:48:43.926481 17514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "85d314529f6e4b13bbedf99781761f65"
format_stamp: "Formatted at 2026-08-06 00:48:43 on dist-test-slave-45dx"
server_key: "c3a4a8d6feffa175f0c32cf5ca95f7b6"
server_key_iv: "191a4a71275b17b3c9b954f5f5bdf6a5"
server_key_version: "encryptionkey@0"
I20260806 00:48:43.930286 17514 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:48:43.932695 17531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:43.933971 17514 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:48:43.934118 17514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "85d314529f6e4b13bbedf99781761f65"
format_stamp: "Formatted at 2026-08-06 00:48:43 on dist-test-slave-45dx"
server_key: "c3a4a8d6feffa175f0c32cf5ca95f7b6"
server_key_iv: "191a4a71275b17b3c9b954f5f5bdf6a5"
server_key_version: "encryptionkey@0"
I20260806 00:48:43.934234 17514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:43.957428 17514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:43.960628 17514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:43.960870 17514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:43.969507 17514 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:41995
I20260806 00:48:43.969542 17625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:41995 every 8 connection(s)
I20260806 00:48:43.970479 17514 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:43.973198 17627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:43.978780 17627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65: Bootstrap starting.
I20260806 00:48:43.979310 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 17514
I20260806 00:48:43.979439 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:48:43.979712 15950 external_mini_cluster.cc:1428] Setting key e98e82fcd4d58b5fdae906dfe0bfdd9c
I20260806 00:48:43.981668 17627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:43.982429 17627 log.cc:824] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:43.984642 17627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65: No bootstrap required, opened a new log
Aug 06 00:48:43 dist-test-slave-45dx krb5kdc[17496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:43.989244 17627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d314529f6e4b13bbedf99781761f65" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41995 } }
I20260806 00:48:43.989560 17627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:43.989621 17627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d314529f6e4b13bbedf99781761f65, State: Initialized, Role: FOLLOWER
I20260806 00:48:43.990104 17627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d314529f6e4b13bbedf99781761f65" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41995 } }
I20260806 00:48:43.990286 17627 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:43.990361 17627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:43.990513 17627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:43.991493 17627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d314529f6e4b13bbedf99781761f65" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41995 } }
I20260806 00:48:43.991807 17627 leader_election.cc:302] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85d314529f6e4b13bbedf99781761f65; no voters:
I20260806 00:48:43.992148 17627 leader_election.cc:288] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:43.992214 17633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:43.992429 17633 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65 [term 1 LEADER]: Becoming Leader. State: Replica: 85d314529f6e4b13bbedf99781761f65, State: Running, Role: LEADER
I20260806 00:48:43.992733 17633 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d314529f6e4b13bbedf99781761f65" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41995 } }
I20260806 00:48:43.993438 17627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:43.994695 17636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85d314529f6e4b13bbedf99781761f65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d314529f6e4b13bbedf99781761f65" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41995 } } }
I20260806 00:48:43.994735 17635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85d314529f6e4b13bbedf99781761f65. Latest consensus state: current_term: 1 leader_uuid: "85d314529f6e4b13bbedf99781761f65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d314529f6e4b13bbedf99781761f65" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41995 } } }
I20260806 00:48:43.994910 17635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:43.995129 17630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:43.981132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56516 (local address 127.15.147.190:41995)
0806 00:48:43.981588 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:43.981604 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:43.981638 (+ 34us) server_negotiation.cc:407] Connection header received
0806 00:48:43.982334 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:43.982360 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:43.982726 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:43.983101 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:43.983933 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:43.984980 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:43.986105 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:43.986542 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:43.986650 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:43.989487 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:43.989494 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:43.989509 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:43.989541 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:43.991543 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:43.992118 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:43.992124 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:43.992129 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:43.992344 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:43.992699 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:43.992704 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:43.992706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:43.993245 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:43.993467 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:43.993738 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 00:48:43.994076 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":141,"threads_started":1}
I20260806 00:48:43.994895 17636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:43.995546 17646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:43.998651 17646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:44.004197 17646 catalog_manager.cc:1347] Generated new cluster ID: 235a1cf4dcc54427a54756d3d66a8532
I20260806 00:48:44.004283 17646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:44.032867 17646 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:44.033857 17646 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:44.043128 17646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65: Generated new TSK 0
I20260806 00:48:44.043830 17646 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
I20260806 00:48:44.089542 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:41995
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:44.216472 17660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:44.216696 17660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:44.216737 17660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:44.220463 17660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:44.220520 17660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:44.220554 17660 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:48:44.220583 17660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:48:44.225468 17660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.147.190:41995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:44.226397 17660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:44.227427 17660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:44.237267 17671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:44.237345 17667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:44.237267 17665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:44.237569 17660 server_base.cc:1034] running on GCE node
I20260806 00:48:44.238059 17660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:44.238828 17660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:44.240048 17660 hybrid_clock.cc:648] HybridClock initialized: now 1785977324240032 us; error 29 us; skew 500 ppm
Aug 06 00:48:44 dist-test-slave-45dx krb5kdc[17496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:44.243050 17660 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:48:44.243968 17660 webserver.cc:460] Webserver started at http://127.15.147.129:38419/ using document root <none> and password file <none>
I20260806 00:48:44.244654 17660 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:44.244755 17660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:44.245002 17660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:44.247272 17660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "834b9db8c8bb46c5ac148a84a8388962"
format_stamp: "Formatted at 2026-08-06 00:48:44 on dist-test-slave-45dx"
server_key: "611964f1c12b91a24b2147f5da2361f4"
server_key_iv: "7bb3f771b7c6fe3c88ce51e74d70f042"
server_key_version: "encryptionkey@0"
I20260806 00:48:44.247908 17660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "834b9db8c8bb46c5ac148a84a8388962"
format_stamp: "Formatted at 2026-08-06 00:48:44 on dist-test-slave-45dx"
server_key: "611964f1c12b91a24b2147f5da2361f4"
server_key_iv: "7bb3f771b7c6fe3c88ce51e74d70f042"
server_key_version: "encryptionkey@0"
I20260806 00:48:44.252008 17660 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 00:48:44.254755 17683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:44.256057 17660 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:48:44.256366 17660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "834b9db8c8bb46c5ac148a84a8388962"
format_stamp: "Formatted at 2026-08-06 00:48:44 on dist-test-slave-45dx"
server_key: "611964f1c12b91a24b2147f5da2361f4"
server_key_iv: "7bb3f771b7c6fe3c88ce51e74d70f042"
server_key_version: "encryptionkey@0"
I20260806 00:48:44.256485 17660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:44.281824 17660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:44.285022 17660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:44.285244 17660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:44.286764 17660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:48:44.286840 17660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:44.286921 17660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:48:44.286971 17660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:48:44 dist-test-slave-45dx krb5kdc[17496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:44.298946 17660 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:42667
I20260806 00:48:44.299016 17844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:42667 every 8 connection(s)
I20260806 00:48:44.300113 17660 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:48:44.301905 17630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.287736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:41879 (local address 127.15.147.190:41995)
0806 00:48:44.287876 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:44.287880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:44.289024 (+ 1144us) server_negotiation.cc:407] Connection header received
0806 00:48:44.290210 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:44.290217 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:44.290297 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:44.290404 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:44.292207 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.292948 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.294282 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.294650 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.294778 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.297618 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:44.297623 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:44.297625 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:44.297665 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:44.299509 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:44.300059 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:44.300066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:44.300071 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:44.300180 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:44.300659 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:44.300663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:44.300665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:44.300924 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:44.301028 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:44.301616 (+ 588us) server_negotiation.cc:299] Negotiation successful
0806 00:48:44.301773 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 00:48:44.302695 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.288280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:41995 (local address 127.15.147.129:41879)
0806 00:48:44.288855 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:44.288902 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:44.289872 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:44.290694 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:44.290706 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:44.291230 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:44.292003 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.292027 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.293087 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.293092 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.294150 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.294161 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.294843 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.294848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.295031 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.295677 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:44.295723 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:44.297419 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:44.299647 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:44.299652 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:44.299664 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:44.299946 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:44.300327 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:44.300331 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:44.300333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:44.300539 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:44.301084 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:44.301089 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:44.301362 (+ 273us) client_negotiation.cc:769] Sending connection context
0806 00:48:44.301519 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 00:48:44.301791 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":138,"threads_started":1}
I20260806 00:48:44.304026 17845 heartbeater.cc:347] Connected to a master server at 127.15.147.190:41995
I20260806 00:48:44.304248 17845 heartbeater.cc:464] Registering TS with master...
I20260806 00:48:44.304767 17845 heartbeater.cc:511] Master 127.15.147.190:41995 requested a full tablet report, sending...
I20260806 00:48:44.306327 17556 ts_manager.cc:194] Registered new tserver with Master: 834b9db8c8bb46c5ac148a84a8388962 (127.15.147.129:42667)
I20260806 00:48:44.307807 17556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:41879
I20260806 00:48:44.308049 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 17660
I20260806 00:48:44.308182 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:48:44.308506 15950 external_mini_cluster.cc:1428] Setting key 4b334edbeb01bb88610b6ddff0094bde
I20260806 00:48:44.314381 17630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.306518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:33659 (local address 127.15.147.190:41995)
0806 00:48:44.306632 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:44.306637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:44.306812 (+ 175us) server_negotiation.cc:407] Connection header received
0806 00:48:44.306906 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:44.306909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:44.306951 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:44.307025 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:44.307434 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.308125 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.309342 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.309682 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.309809 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.311065 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:44.311069 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:44.311071 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:44.311095 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:44.312728 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:44.313176 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:44.313180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:44.313182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:44.313265 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:44.313559 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:44.313563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:44.313564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:44.313739 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:44.313853 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:44.314119 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 00:48:44.314227 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:48:44.314419 17856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.306420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:41995 (local address 127.15.147.129:33659)
0806 00:48:44.306724 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:44.306737 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:44.306827 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:44.307051 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:44.307053 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:44.307225 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:44.307327 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.307331 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.308329 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.308333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.309209 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.309224 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.309939 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.309943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.310039 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.310572 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:44.310593 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:44.310923 (+ 330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:44.312860 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:44.312864 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:44.312866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:44.313073 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:44.313367 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:44.313371 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:44.313373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:44.313466 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:44.313854 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:44.313858 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:44.313956 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 00:48:44.314065 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 00:48:44.314213 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":118,"threads_started":1}
WARNING: no policy specified for kudu/127.15.147.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
I20260806 00:48:44.337421 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:41995
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:44.451254 17860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:44.451526 17860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:44.451601 17860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:44.455570 17860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:44.455663 17860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:44.455731 17860 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:48:44.455809 17860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:48:44.461066 17860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.147.190:41995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:44.462085 17860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:44.463099 17860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:44.472282 17869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:44.472370 17870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:44.472370 17872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:44.472477 17860 server_base.cc:1034] running on GCE node
I20260806 00:48:44.472971 17860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:44.473555 17860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:44.474742 17860 hybrid_clock.cc:648] HybridClock initialized: now 1785977324474723 us; error 35 us; skew 500 ppm
Aug 06 00:48:44 dist-test-slave-45dx krb5kdc[17496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:44.477552 17860 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:48:44.478361 17860 webserver.cc:460] Webserver started at http://127.15.147.130:38793/ using document root <none> and password file <none>
I20260806 00:48:44.478909 17860 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:44.479000 17860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:44.479228 17860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:44.481333 17860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "9ac000c0683a4c3da82e3caf2b130adb"
format_stamp: "Formatted at 2026-08-06 00:48:44 on dist-test-slave-45dx"
server_key: "4a840ded7560c23301b82bd7f66ca302"
server_key_iv: "81aef18052105ce3c31d78382761ae60"
server_key_version: "encryptionkey@0"
I20260806 00:48:44.481937 17860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "9ac000c0683a4c3da82e3caf2b130adb"
format_stamp: "Formatted at 2026-08-06 00:48:44 on dist-test-slave-45dx"
server_key: "4a840ded7560c23301b82bd7f66ca302"
server_key_iv: "81aef18052105ce3c31d78382761ae60"
server_key_version: "encryptionkey@0"
I20260806 00:48:44.485922 17860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:48:44.488426 17884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:44.489614 17860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:48:44.489754 17860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "9ac000c0683a4c3da82e3caf2b130adb"
format_stamp: "Formatted at 2026-08-06 00:48:44 on dist-test-slave-45dx"
server_key: "4a840ded7560c23301b82bd7f66ca302"
server_key_iv: "81aef18052105ce3c31d78382761ae60"
server_key_version: "encryptionkey@0"
I20260806 00:48:44.489892 17860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:44.507783 17860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:44.510536 17860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:44.510740 17860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:44.512148 17860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:48:44.512225 17860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:44.512332 17860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:48:44.512391 17860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:48:44 dist-test-slave-45dx krb5kdc[17496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:44.523859 17860 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:44733
I20260806 00:48:44.524174 18052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:44733 every 8 connection(s)
I20260806 00:48:44.525050 17860 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:48:44.527102 17630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.512998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:58707 (local address 127.15.147.190:41995)
0806 00:48:44.513126 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:44.513130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:44.514168 (+ 1038us) server_negotiation.cc:407] Connection header received
0806 00:48:44.515153 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:44.515157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:44.515219 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:44.515306 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:44.516743 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.517616 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.518930 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.519346 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.519472 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.522913 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:44.522917 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:44.522920 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:44.522949 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:44.524794 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:44.525358 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:44.525363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:44.525365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:44.525472 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:44.525882 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:44.525885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:44.525887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:44.526133 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:44.526231 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:44.526775 (+ 544us) server_negotiation.cc:299] Negotiation successful
0806 00:48:44.526916 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:48:44.527861 17895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.513545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:41995 (local address 127.15.147.130:58707)
0806 00:48:44.514021 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:44.514062 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:44.514945 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:44.515597 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:44.515604 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:44.516002 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:44.516596 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.516616 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.517764 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.517767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.518792 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.518803 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.519551 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.519556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.519705 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.520400 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:44.520418 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:44.522733 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:44.524906 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:44.524911 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:44.524922 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:44.525256 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:44.525604 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:44.525607 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:44.525609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:44.525787 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:44.526237 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:44.526246 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:44.526531 (+ 285us) client_negotiation.cc:769] Sending connection context
0806 00:48:44.526727 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 00:48:44.527051 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":137,"threads_started":1}
I20260806 00:48:44.529086 18053 heartbeater.cc:347] Connected to a master server at 127.15.147.190:41995
I20260806 00:48:44.529234 18053 heartbeater.cc:464] Registering TS with master...
I20260806 00:48:44.529830 18053 heartbeater.cc:511] Master 127.15.147.190:41995 requested a full tablet report, sending...
I20260806 00:48:44.530974 17555 ts_manager.cc:194] Registered new tserver with Master: 9ac000c0683a4c3da82e3caf2b130adb (127.15.147.130:44733)
I20260806 00:48:44.531788 17555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:58707
I20260806 00:48:44.535000 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 17860
I20260806 00:48:44.535128 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:48:44.535818 15950 external_mini_cluster.cc:1428] Setting key 60ae27c75f4ae8192b9201fddc468928
I20260806 00:48:44.539731 17630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.531595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:58823 (local address 127.15.147.190:41995)
0806 00:48:44.531728 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:44.531731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:44.531995 (+ 264us) server_negotiation.cc:407] Connection header received
0806 00:48:44.532393 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:44.532395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:44.532435 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:44.532500 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:44.533054 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.533549 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.534474 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.534709 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.534829 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.536148 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:44.536151 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:44.536153 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:44.536176 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:44.537729 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:44.538276 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:44.538280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:44.538282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:44.538409 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:44.538751 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:44.538753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:44.538755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:44.538990 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:44.539081 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:44.539393 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 00:48:44.539523 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:48:44.539772 18060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.531511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:41995 (local address 127.15.147.130:58823)
0806 00:48:44.531880 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:44.531893 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:44.532017 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:44.532545 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:44.532548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:44.532785 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:44.532936 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.532944 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.533680 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.533684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.534283 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.534293 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.534848 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.534852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.534966 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.535565 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:44.535582 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:44.536014 (+ 432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:44.537848 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:44.537854 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:44.537859 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:44.538155 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:44.538508 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:44.538515 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:44.538520 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:44.538628 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:44.539095 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:44.539101 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:44.539218 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 00:48:44.539338 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 00:48:44.539522 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for kudu/127.15.147.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
I20260806 00:48:44.565758 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:41995
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:44.680140 18065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:44.680486 18065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:44.680576 18065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:44.684530 18065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:44.684612 18065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:44.684679 18065 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:48:44.684756 18065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:48:44.689869 18065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.147.190:41995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:44.690865 18065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:44.691860 18065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:44.702243 18074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:44.702250 18065 server_base.cc:1034] running on GCE node
W20260806 00:48:44.702215 18080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:44.702215 18073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:44.702884 18065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:44.703480 18065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:44.704665 18065 hybrid_clock.cc:648] HybridClock initialized: now 1785977324704632 us; error 43 us; skew 500 ppm
Aug 06 00:48:44 dist-test-slave-45dx krb5kdc[17496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:44.707464 18065 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:48:44.708292 18065 webserver.cc:460] Webserver started at http://127.15.147.131:35761/ using document root <none> and password file <none>
I20260806 00:48:44.708886 18065 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:44.708989 18065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:44.709224 18065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:44.711194 18065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "5301f9aa1c144403a3e7adc74157dedc"
format_stamp: "Formatted at 2026-08-06 00:48:44 on dist-test-slave-45dx"
server_key: "ca5756cd7c56371140468989416c479a"
server_key_iv: "ae67ab41519d5e22b47926b0589d6cd3"
server_key_version: "encryptionkey@0"
I20260806 00:48:44.711750 18065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "5301f9aa1c144403a3e7adc74157dedc"
format_stamp: "Formatted at 2026-08-06 00:48:44 on dist-test-slave-45dx"
server_key: "ca5756cd7c56371140468989416c479a"
server_key_iv: "ae67ab41519d5e22b47926b0589d6cd3"
server_key_version: "encryptionkey@0"
I20260806 00:48:44.715407 18065 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:48:44.717754 18088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:44.718892 18065 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:48:44.719031 18065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "5301f9aa1c144403a3e7adc74157dedc"
format_stamp: "Formatted at 2026-08-06 00:48:44 on dist-test-slave-45dx"
server_key: "ca5756cd7c56371140468989416c479a"
server_key_iv: "ae67ab41519d5e22b47926b0589d6cd3"
server_key_version: "encryptionkey@0"
I20260806 00:48:44.719135 18065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:44.750844 18065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:44.754359 18065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:44.754586 18065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:44.756093 18065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:48:44.756168 18065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:44.756244 18065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:48:44.756289 18065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:48:44 dist-test-slave-45dx krb5kdc[17496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977324, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:44.768137 18065 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:46637
I20260806 00:48:44.768219 18252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:46637 every 8 connection(s)
I20260806 00:48:44.769048 18065 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:48:44.770812 17630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.757030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:52111 (local address 127.15.147.190:41995)
0806 00:48:44.757192 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:44.757197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:44.758102 (+ 905us) server_negotiation.cc:407] Connection header received
0806 00:48:44.759250 (+ 1148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:44.759256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:44.759327 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:44.759405 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:44.761201 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.761869 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.763163 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.763419 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.763537 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.766970 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:44.766974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:44.766977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:44.767011 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:44.768642 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:44.769157 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:44.769162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:44.769164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:44.769262 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:44.769699 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:44.769702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:44.769704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:44.769934 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:44.770033 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:44.770513 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 00:48:44.770639 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 00:48:44.771551 18101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.757452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:41995 (local address 127.15.147.131:52111)
0806 00:48:44.757926 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:44.757979 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:44.758953 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:44.759887 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:44.759897 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:44.760411 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:44.761055 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.761071 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.762002 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.762005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.763034 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.763044 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.763593 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.763597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.763774 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.764566 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:44.764585 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:44.766752 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:44.768760 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:44.768766 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:44.768779 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:44.769048 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:44.769377 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:44.769381 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:44.769384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:44.769571 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:44.770029 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:44.770035 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:44.770270 (+ 235us) client_negotiation.cc:769] Sending connection context
0806 00:48:44.770466 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 00:48:44.770769 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":134,"threads_started":1}
I20260806 00:48:44.772841 18257 heartbeater.cc:347] Connected to a master server at 127.15.147.190:41995
I20260806 00:48:44.773008 18257 heartbeater.cc:464] Registering TS with master...
I20260806 00:48:44.773170 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 18065
I20260806 00:48:44.773301 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:48:44.773617 18257 heartbeater.cc:511] Master 127.15.147.190:41995 requested a full tablet report, sending...
I20260806 00:48:44.773643 15950 external_mini_cluster.cc:1428] Setting key e07d7ce7567c1d3b6a6ca3a36b466db0
I20260806 00:48:44.774729 17556 ts_manager.cc:194] Registered new tserver with Master: 5301f9aa1c144403a3e7adc74157dedc (127.15.147.131:46637)
I20260806 00:48:44.775475 17556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:52111
I20260806 00:48:44.777139 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:48:44.782466 18267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.775086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:41995 (local address 127.15.147.131:46667)
0806 00:48:44.775373 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:44.775387 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:44.775484 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:44.775755 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:44.775759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:44.775997 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:44.776124 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.776133 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.777131 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.777134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.777764 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.777773 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.778235 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.778238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.778338 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.778881 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:44.778893 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:44.779195 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:44.780950 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:44.780953 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:44.780955 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:44.781154 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:44.781434 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:44.781440 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:44.781443 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:44.781534 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:44.781909 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:44.781912 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:44.782017 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 00:48:44.782127 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 00:48:44.782279 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":116,"threads_started":1}
I20260806 00:48:44.782557 17630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.775161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:46667 (local address 127.15.147.190:41995)
0806 00:48:44.775291 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:44.775295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:44.775459 (+ 164us) server_negotiation.cc:407] Connection header received
0806 00:48:44.775588 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:44.775592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:44.775636 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:44.775707 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:44.776287 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.776998 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.777888 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.778120 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.778229 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.779336 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:44.779339 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:44.779342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:44.779371 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:44.780827 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:44.781254 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:44.781257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:44.781259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:44.781339 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:44.781627 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:44.781632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:44.781633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:44.781810 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:44.781918 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:44.782288 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 00:48:44.782390 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 00:48:44 dist-test-slave-45dx krb5kdc[17496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977324, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 00:48:44 dist-test-slave-45dx krb5kdc[17496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977324, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:44.804575 17630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.796602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56518 (local address 127.15.147.190:41995)
0806 00:48:44.796738 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:44.796742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:44.796753 (+ 11us) server_negotiation.cc:407] Connection header received
0806 00:48:44.796789 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:44.796792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:44.796828 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:44.796919 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:44.797644 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.798156 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.798990 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.799201 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.799318 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.801666 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:44.801674 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:44.801676 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:44.801698 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:44.802883 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:44.803337 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:44.803340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:44.803341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:44.803415 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:44.803709 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:44.803713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:44.803715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:44.803887 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:44.804022 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:44.804257 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 00:48:44.804414 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 00:48:44.807353 17556 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:48:44.810367 17556 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:48:44.824455 18288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.818455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:42667 (local address 127.0.0.1:52700)
0806 00:48:44.818814 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:44.818864 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:44.819008 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:44.819783 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:44.819822 (+ 39us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:44.819848 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:44.819975 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.819983 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.821454 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.821457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.822829 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.822838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.823873 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.823876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.823993 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.824030 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 00:48:44.824164 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 00:48:44.824284 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":1,"thread_start_us":82,"threads_started":1}
I20260806 00:48:44.824455 18289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.818589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52700 (local address 127.15.147.129:42667)
0806 00:48:44.818956 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:44.818962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:44.819004 (+ 42us) server_negotiation.cc:407] Connection header received
0806 00:48:44.819228 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:44.819235 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:44.819453 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:44.819604 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:44.820122 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.821316 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.822962 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.823749 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.823886 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.824025 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:44.824194 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 00:48:44.824328 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":72,"threads_started":1}
I20260806 00:48:44.824887 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.818879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:44733 (local address 127.0.0.1:35886)
0806 00:48:44.819466 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:44.819481 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:44.819572 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:44.820604 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:44.820607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:44.820625 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:44.820738 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.820744 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.822009 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.822016 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.823197 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.823208 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.824472 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.824476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.824563 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.824586 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:48:44.824663 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 00:48:44.824746 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":110,"threads_started":1}
I20260806 00:48:44.825241 18295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.819666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55914 (local address 127.15.147.131:46637)
0806 00:48:44.820455 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:44.820464 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:44.820482 (+ 18us) server_negotiation.cc:407] Connection header received
0806 00:48:44.820540 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:44.820544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:44.820752 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:44.820891 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:44.821224 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.822363 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.823835 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.824656 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.824787 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.824905 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:44.825016 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 00:48:44.825098 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":125,"threads_started":1}
I20260806 00:48:44.825425 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.819273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35886 (local address 127.15.147.130:44733)
0806 00:48:44.820129 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:44.820142 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:44.820161 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:48:44.820232 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:44.820237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:44.820464 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:44.820612 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:44.820867 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.821898 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.823309 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.824125 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.824260 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.824404 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:44.825174 (+ 770us) server_negotiation.cc:299] Negotiation successful
0806 00:48:44.825273 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":121,"threads_started":1}
I20260806 00:48:44.825176 18291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.819206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:46637 (local address 127.0.0.1:55914)
0806 00:48:44.819884 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:44.819897 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:44.820099 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:44.821015 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:44.821017 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:44.821031 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:44.821110 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.821114 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.822551 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.822555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.823699 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.823710 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.824755 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.824759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.824864 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.824889 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 00:48:44.824980 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 00:48:44.825057 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":2,"thread_start_us":125,"threads_started":1}
I20260806 00:48:44.827924 18163 tablet_service.cc:1467] Processing CreateTablet for tablet aca8df96bd9e4d18a0f6dea52d9c0485 (DEFAULT_TABLE table=test-table [id=c85b4f6e389545f6b933cd82eacdb1d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:44.827770 17745 tablet_service.cc:1467] Processing CreateTablet for tablet aca8df96bd9e4d18a0f6dea52d9c0485 (DEFAULT_TABLE table=test-table [id=c85b4f6e389545f6b933cd82eacdb1d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:44.828961 18163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aca8df96bd9e4d18a0f6dea52d9c0485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:44.828964 17745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aca8df96bd9e4d18a0f6dea52d9c0485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:44.828758 17954 tablet_service.cc:1467] Processing CreateTablet for tablet aca8df96bd9e4d18a0f6dea52d9c0485 (DEFAULT_TABLE table=test-table [id=c85b4f6e389545f6b933cd82eacdb1d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:44.829681 17954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aca8df96bd9e4d18a0f6dea52d9c0485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:44.836193 18298 tablet_bootstrap.cc:492] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc: Bootstrap starting.
I20260806 00:48:44.837549 18300 tablet_bootstrap.cc:492] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb: Bootstrap starting.
I20260806 00:48:44.838749 18298 tablet_bootstrap.cc:654] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:44.839419 18300 tablet_bootstrap.cc:654] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:44.839556 18298 log.cc:824] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:44.840224 18300 log.cc:824] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:44.840806 18299 tablet_bootstrap.cc:492] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962: Bootstrap starting.
I20260806 00:48:44.841939 18298 tablet_bootstrap.cc:492] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc: No bootstrap required, opened a new log
I20260806 00:48:44.842140 18298 ts_tablet_manager.cc:1397] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:48:44.842329 18300 tablet_bootstrap.cc:492] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb: No bootstrap required, opened a new log
I20260806 00:48:44.842520 18300 ts_tablet_manager.cc:1397] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 00:48:44.843019 18299 tablet_bootstrap.cc:654] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:44.843837 18299 log.cc:824] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:44.845834 18299 tablet_bootstrap.cc:492] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962: No bootstrap required, opened a new log
I20260806 00:48:44.845775 18300 raft_consensus.cc:348] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } }
I20260806 00:48:44.846014 18299 ts_tablet_manager.cc:1397] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:48:44.846067 18300 raft_consensus.cc:374] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:44.846143 18300 raft_consensus.cc:729] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ac000c0683a4c3da82e3caf2b130adb, State: Initialized, Role: FOLLOWER
I20260806 00:48:44.846213 18298 raft_consensus.cc:348] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } }
I20260806 00:48:44.846459 18298 raft_consensus.cc:374] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:44.846519 18298 raft_consensus.cc:729] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5301f9aa1c144403a3e7adc74157dedc, State: Initialized, Role: FOLLOWER
I20260806 00:48:44.846621 18300 consensus_queue.cc:260] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } }
I20260806 00:48:44.846933 18298 consensus_queue.cc:260] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } }
I20260806 00:48:44.847328 18053 heartbeater.cc:503] Master 127.15.147.190:41995 was elected leader, sending a full tablet report...
I20260806 00:48:44.848057 18300 ts_tablet_manager.cc:1428] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 00:48:44.848109 18298 ts_tablet_manager.cc:1428] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 00:48:44.848487 18257 heartbeater.cc:503] Master 127.15.147.190:41995 was elected leader, sending a full tablet report...
I20260806 00:48:44.851040 18299 raft_consensus.cc:348] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } }
I20260806 00:48:44.851307 18299 raft_consensus.cc:374] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:44.851372 18299 raft_consensus.cc:729] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 834b9db8c8bb46c5ac148a84a8388962, State: Initialized, Role: FOLLOWER
I20260806 00:48:44.851838 18299 consensus_queue.cc:260] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } }
I20260806 00:48:44.852514 17845 heartbeater.cc:503] Master 127.15.147.190:41995 was elected leader, sending a full tablet report...
I20260806 00:48:44.852811 18299 ts_tablet_manager.cc:1428] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 00:48:44.874938 18306 raft_consensus.cc:482] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:48:44.875133 18306 raft_consensus.cc:504] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } }
I20260806 00:48:44.876288 18306 leader_election.cc:288] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5301f9aa1c144403a3e7adc74157dedc (127.15.147.131:46637), 9ac000c0683a4c3da82e3caf2b130adb (127.15.147.130:44733)
I20260806 00:48:44.880787 18295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.876834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:48015 (local address 127.15.147.131:46637)
0806 00:48:44.876996 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:44.877000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:44.877290 (+ 290us) server_negotiation.cc:407] Connection header received
0806 00:48:44.877417 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:44.877420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:44.877466 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:44.877545 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:44.877842 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.878357 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.879768 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.880174 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.880267 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.880379 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:44.880564 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 00:48:44.880649 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 00:48:44.880815 18307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.876622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:46637 (local address 127.15.147.129:48015)
0806 00:48:44.877177 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:44.877191 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:44.877295 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:44.877576 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:44.877578 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:44.877594 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:44.877725 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.877732 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.878507 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.878510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.879647 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.879657 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.880320 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.880327 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.880427 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.880450 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:48:44.880532 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 00:48:44.880611 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":18,"thread_start_us":126,"threads_started":1}
I20260806 00:48:44.880851 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.876755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:55763 (local address 127.15.147.130:44733)
0806 00:48:44.876915 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:44.876921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:44.877143 (+ 222us) server_negotiation.cc:407] Connection header received
0806 00:48:44.877240 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:44.877245 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:44.877304 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:44.877383 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:44.877729 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.878346 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.879797 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.880205 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.880381 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.880453 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:44.880646 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 00:48:44.880731 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:48:44.880834 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.876571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:44733 (local address 127.15.147.129:55763)
0806 00:48:44.877039 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:44.877051 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:44.877152 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:44.877417 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:44.877420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:44.877440 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:44.877597 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.877605 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.878486 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.878490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.879650 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:44.879660 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.880375 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.880379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:44.880486 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.880529 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 00:48:44.880609 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 00:48:44.880670 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":54,"thread_start_us":157,"threads_started":1}
I20260806 00:48:44.881641 18189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca8df96bd9e4d18a0f6dea52d9c0485" candidate_uuid: "834b9db8c8bb46c5ac148a84a8388962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5301f9aa1c144403a3e7adc74157dedc" is_pre_election: true
I20260806 00:48:44.881899 17981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca8df96bd9e4d18a0f6dea52d9c0485" candidate_uuid: "834b9db8c8bb46c5ac148a84a8388962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac000c0683a4c3da82e3caf2b130adb" is_pre_election: true
I20260806 00:48:44.882021 18189 raft_consensus.cc:2454] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 834b9db8c8bb46c5ac148a84a8388962 in term 0.
I20260806 00:48:44.882164 17981 raft_consensus.cc:2454] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 834b9db8c8bb46c5ac148a84a8388962 in term 0.
I20260806 00:48:44.882525 17685 leader_election.cc:302] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5301f9aa1c144403a3e7adc74157dedc, 834b9db8c8bb46c5ac148a84a8388962; no voters:
I20260806 00:48:44.882815 18306 raft_consensus.cc:2781] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:48:44.882918 18306 raft_consensus.cc:482] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:48:44.882966 18306 raft_consensus.cc:3037] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:44.884042 18306 raft_consensus.cc:504] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } }
I20260806 00:48:44.884415 18306 leader_election.cc:288] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [CANDIDATE]: Term 1 election: Requested vote from peers 5301f9aa1c144403a3e7adc74157dedc (127.15.147.131:46637), 9ac000c0683a4c3da82e3caf2b130adb (127.15.147.130:44733)
I20260806 00:48:44.884729 18189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca8df96bd9e4d18a0f6dea52d9c0485" candidate_uuid: "834b9db8c8bb46c5ac148a84a8388962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5301f9aa1c144403a3e7adc74157dedc"
I20260806 00:48:44.884799 17981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aca8df96bd9e4d18a0f6dea52d9c0485" candidate_uuid: "834b9db8c8bb46c5ac148a84a8388962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac000c0683a4c3da82e3caf2b130adb"
I20260806 00:48:44.884864 18189 raft_consensus.cc:3037] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:44.884927 17981 raft_consensus.cc:3037] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:44.886438 18189 raft_consensus.cc:2454] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 834b9db8c8bb46c5ac148a84a8388962 in term 1.
I20260806 00:48:44.886816 17685 leader_election.cc:302] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5301f9aa1c144403a3e7adc74157dedc, 834b9db8c8bb46c5ac148a84a8388962; no voters:
I20260806 00:48:44.887068 18306 raft_consensus.cc:2781] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:44.887316 18306 raft_consensus.cc:686] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [term 1 LEADER]: Becoming Leader. State: Replica: 834b9db8c8bb46c5ac148a84a8388962, State: Running, Role: LEADER
I20260806 00:48:44.887382 17981 raft_consensus.cc:2454] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 834b9db8c8bb46c5ac148a84a8388962 in term 1.
I20260806 00:48:44.887701 18306 consensus_queue.cc:237] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } }
I20260806 00:48:44.891125 17556 catalog_manager.cc:5495] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 reported cstate change: term changed from 0 to 1, leader changed from <none> to 834b9db8c8bb46c5ac148a84a8388962 (127.15.147.129). New cstate: current_term: 1 leader_uuid: "834b9db8c8bb46c5ac148a84a8388962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } health_report { overall_health: UNKNOWN } } }
I20260806 00:48:44.913683 18289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:44.907190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52708 (local address 127.15.147.129:42667)
0806 00:48:44.907992 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:44.907999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:44.908016 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:48:44.908117 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:44.908120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:44.908192 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:44.908325 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:44.908761 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.909707 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.911423 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:44.911829 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:44.911957 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:44.912519 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:44.913058 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:44.913177 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:44.913413 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 00:48:44.913483 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673}
I20260806 00:48:44.922921 17981 raft_consensus.cc:1261] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb [term 1 FOLLOWER]: Refusing update from remote peer 834b9db8c8bb46c5ac148a84a8388962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:44.923818 18309 consensus_queue.cc:1035] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:44.924860 18189 raft_consensus.cc:1261] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc [term 1 FOLLOWER]: Refusing update from remote peer 834b9db8c8bb46c5ac148a84a8388962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:44.925597 18309 consensus_queue.cc:1035] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:44.935680 18314 mvcc.cc:205] Tried to move back new op lower bound from 7315363122873872384 to 7315363122742960128. Current Snapshot: MvccSnapshot[applied={T|T < 7315363122873872384}]
I20260806 00:48:44.955502 17555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:48:44.955859 17555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:48:44.959987 17555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65: Sending DeleteTablet for 3 replicas of tablet aca8df96bd9e4d18a0f6dea52d9c0485
I20260806 00:48:44.961563 18163 tablet_service.cc:1514] Processing DeleteTablet for tablet aca8df96bd9e4d18a0f6dea52d9c0485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:48:44 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:55914
I20260806 00:48:44.961768 17745 tablet_service.cc:1514] Processing DeleteTablet for tablet aca8df96bd9e4d18a0f6dea52d9c0485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:48:44 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:52700
I20260806 00:48:44.961711 17954 tablet_service.cc:1514] Processing DeleteTablet for tablet aca8df96bd9e4d18a0f6dea52d9c0485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:48:44 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:35886
I20260806 00:48:44.962852 18333 tablet_replica.cc:330] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc: stopping tablet replica
I20260806 00:48:44.962965 18334 tablet_replica.cc:330] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb: stopping tablet replica
I20260806 00:48:44.963500 18333 raft_consensus.cc:2229] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:48:44.963580 18334 raft_consensus.cc:2229] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:48:44.964160 18334 raft_consensus.cc:2258] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:48:44.964468 18333 raft_consensus.cc:2258] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:48:44.964674 17555 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:48:44.965286 17555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:48:44.968513 18334 ts_tablet_manager.cc:1905] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:48:44.969118 18335 tablet_replica.cc:330] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962: stopping tablet replica
I20260806 00:48:44.970762 18333 ts_tablet_manager.cc:1905] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:48:44.971321 18335 raft_consensus.cc:2229] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:48:44.972043 18335 raft_consensus.cc:2258] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:48:44.974061 17954 tablet_service.cc:1467] Processing CreateTablet for tablet 5388334122e84c5ca7d35c8b087fcc66 (DEFAULT_TABLE table=test-table [id=a7597e9da7c64ae6b4b103e5e0e895f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:44.974440 17954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5388334122e84c5ca7d35c8b087fcc66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:44.976694 18335 ts_tablet_manager.cc:1905] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:48:44.976725 18163 tablet_service.cc:1467] Processing CreateTablet for tablet 5388334122e84c5ca7d35c8b087fcc66 (DEFAULT_TABLE table=test-table [id=a7597e9da7c64ae6b4b103e5e0e895f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:44.977166 18163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5388334122e84c5ca7d35c8b087fcc66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:44.977645 18333 ts_tablet_manager.cc:1918] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:48:44.977842 18333 log.cc:1196] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/aca8df96bd9e4d18a0f6dea52d9c0485
I20260806 00:48:44.978504 18333 ts_tablet_manager.cc:1939] T aca8df96bd9e4d18a0f6dea52d9c0485 P 5301f9aa1c144403a3e7adc74157dedc: Deleting consensus metadata
I20260806 00:48:44.980980 17537 catalog_manager.cc:4841] TS 5301f9aa1c144403a3e7adc74157dedc (127.15.147.131:46637): tablet aca8df96bd9e4d18a0f6dea52d9c0485 (table test-table [id=c85b4f6e389545f6b933cd82eacdb1d7]) successfully deleted
I20260806 00:48:44.982146 18335 ts_tablet_manager.cc:1918] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:48:44.982178 17745 tablet_service.cc:1467] Processing CreateTablet for tablet 5388334122e84c5ca7d35c8b087fcc66 (DEFAULT_TABLE table=test-table [id=a7597e9da7c64ae6b4b103e5e0e895f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:44.982326 18335 log.cc:1196] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/wal/wals/aca8df96bd9e4d18a0f6dea52d9c0485
I20260806 00:48:44.982613 17745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5388334122e84c5ca7d35c8b087fcc66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:44.982883 18335 ts_tablet_manager.cc:1939] T aca8df96bd9e4d18a0f6dea52d9c0485 P 834b9db8c8bb46c5ac148a84a8388962: Deleting consensus metadata
I20260806 00:48:44.986105 18298 tablet_bootstrap.cc:492] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc: Bootstrap starting.
I20260806 00:48:44.986383 18299 tablet_bootstrap.cc:492] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962: Bootstrap starting.
I20260806 00:48:44.986979 18334 ts_tablet_manager.cc:1918] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:48:44.987167 18334 log.cc:1196] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/aca8df96bd9e4d18a0f6dea52d9c0485
I20260806 00:48:44.987618 18334 ts_tablet_manager.cc:1939] T aca8df96bd9e4d18a0f6dea52d9c0485 P 9ac000c0683a4c3da82e3caf2b130adb: Deleting consensus metadata
I20260806 00:48:44.988113 18299 tablet_bootstrap.cc:654] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:44.988281 18298 tablet_bootstrap.cc:654] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:44.989866 17536 catalog_manager.cc:4841] TS 9ac000c0683a4c3da82e3caf2b130adb (127.15.147.130:44733): tablet aca8df96bd9e4d18a0f6dea52d9c0485 (table test-table [id=c85b4f6e389545f6b933cd82eacdb1d7]) successfully deleted
I20260806 00:48:44.990272 18300 tablet_bootstrap.cc:492] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb: Bootstrap starting.
I20260806 00:48:44.991732 18300 tablet_bootstrap.cc:654] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:44.992100 18298 tablet_bootstrap.cc:492] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc: No bootstrap required, opened a new log
I20260806 00:48:44.992479 18298 ts_tablet_manager.cc:1397] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260806 00:48:44.994691 18299 tablet_bootstrap.cc:492] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962: No bootstrap required, opened a new log
I20260806 00:48:44.994871 18299 ts_tablet_manager.cc:1397] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260806 00:48:44.995653 18300 tablet_bootstrap.cc:492] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb: No bootstrap required, opened a new log
I20260806 00:48:44.995764 18300 ts_tablet_manager.cc:1397] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 00:48:44.995848 18299 raft_consensus.cc:348] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } }
I20260806 00:48:44.996052 18299 raft_consensus.cc:374] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:44.996122 18299 raft_consensus.cc:729] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 834b9db8c8bb46c5ac148a84a8388962, State: Initialized, Role: FOLLOWER
I20260806 00:48:44.996325 18299 consensus_queue.cc:260] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } }
I20260806 00:48:44.996681 18299 ts_tablet_manager.cc:1428] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 00:48:44.997061 18300 raft_consensus.cc:348] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } }
I20260806 00:48:44.997192 18300 raft_consensus.cc:374] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:44.997234 18300 raft_consensus.cc:729] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ac000c0683a4c3da82e3caf2b130adb, State: Initialized, Role: FOLLOWER
I20260806 00:48:44.997475 18300 consensus_queue.cc:260] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } }
I20260806 00:48:44.997936 18300 ts_tablet_manager.cc:1428] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260806 00:48:44.999125 17540 catalog_manager.cc:4841] TS 834b9db8c8bb46c5ac148a84a8388962 (127.15.147.129:42667): tablet aca8df96bd9e4d18a0f6dea52d9c0485 (table test-table [id=c85b4f6e389545f6b933cd82eacdb1d7]) successfully deleted
I20260806 00:48:44.998430 18298 raft_consensus.cc:348] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } }
I20260806 00:48:44.999867 18298 raft_consensus.cc:374] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:45.000025 18298 raft_consensus.cc:729] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5301f9aa1c144403a3e7adc74157dedc, State: Initialized, Role: FOLLOWER
I20260806 00:48:45.000469 18298 consensus_queue.cc:260] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } }
I20260806 00:48:45.002230 18298 ts_tablet_manager.cc:1428] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
W20260806 00:48:45.020514 18260 tablet.cc:2369] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:48:45.024050 18306 raft_consensus.cc:482] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:48:45.024222 18306 raft_consensus.cc:504] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } }
I20260806 00:48:45.024966 18306 leader_election.cc:288] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ac000c0683a4c3da82e3caf2b130adb (127.15.147.130:44733), 5301f9aa1c144403a3e7adc74157dedc (127.15.147.131:46637)
I20260806 00:48:45.025439 17981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5388334122e84c5ca7d35c8b087fcc66" candidate_uuid: "834b9db8c8bb46c5ac148a84a8388962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac000c0683a4c3da82e3caf2b130adb" is_pre_election: true
I20260806 00:48:45.025630 18189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5388334122e84c5ca7d35c8b087fcc66" candidate_uuid: "834b9db8c8bb46c5ac148a84a8388962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5301f9aa1c144403a3e7adc74157dedc" is_pre_election: true
I20260806 00:48:45.025712 17981 raft_consensus.cc:2454] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 834b9db8c8bb46c5ac148a84a8388962 in term 0.
I20260806 00:48:45.025874 18189 raft_consensus.cc:2454] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 834b9db8c8bb46c5ac148a84a8388962 in term 0.
W20260806 00:48:45.026338 18054 tablet.cc:2369] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:48:45.026397 17684 leader_election.cc:302] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 834b9db8c8bb46c5ac148a84a8388962, 9ac000c0683a4c3da82e3caf2b130adb; no voters:
I20260806 00:48:45.026669 18306 raft_consensus.cc:2781] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:48:45.026772 18306 raft_consensus.cc:482] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:48:45.026850 18306 raft_consensus.cc:3037] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:45.029956 18306 raft_consensus.cc:504] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } }
I20260806 00:48:45.030573 18306 leader_election.cc:288] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [CANDIDATE]: Term 1 election: Requested vote from peers 9ac000c0683a4c3da82e3caf2b130adb (127.15.147.130:44733), 5301f9aa1c144403a3e7adc74157dedc (127.15.147.131:46637)
I20260806 00:48:45.031098 17981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5388334122e84c5ca7d35c8b087fcc66" candidate_uuid: "834b9db8c8bb46c5ac148a84a8388962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ac000c0683a4c3da82e3caf2b130adb"
I20260806 00:48:45.031127 18189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5388334122e84c5ca7d35c8b087fcc66" candidate_uuid: "834b9db8c8bb46c5ac148a84a8388962" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5301f9aa1c144403a3e7adc74157dedc"
I20260806 00:48:45.031288 17981 raft_consensus.cc:3037] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:45.031292 18189 raft_consensus.cc:3037] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:45.032756 17981 raft_consensus.cc:2454] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 834b9db8c8bb46c5ac148a84a8388962 in term 1.
I20260806 00:48:45.033217 17684 leader_election.cc:302] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 834b9db8c8bb46c5ac148a84a8388962, 9ac000c0683a4c3da82e3caf2b130adb; no voters:
I20260806 00:48:45.033392 18189 raft_consensus.cc:2454] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 834b9db8c8bb46c5ac148a84a8388962 in term 1.
I20260806 00:48:45.033563 18306 raft_consensus.cc:2781] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:45.033759 18306 raft_consensus.cc:686] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [term 1 LEADER]: Becoming Leader. State: Replica: 834b9db8c8bb46c5ac148a84a8388962, State: Running, Role: LEADER
I20260806 00:48:45.034082 18306 consensus_queue.cc:237] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } }
I20260806 00:48:45.037058 17556 catalog_manager.cc:5495] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 reported cstate change: term changed from 0 to 1, leader changed from <none> to 834b9db8c8bb46c5ac148a84a8388962 (127.15.147.129). New cstate: current_term: 1 leader_uuid: "834b9db8c8bb46c5ac148a84a8388962" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } health_report { overall_health: UNKNOWN } } }
W20260806 00:48:45.053961 17846 tablet.cc:2369] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 00:48:45 dist-test-slave-45dx krb5kdc[17496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977323, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:45.089987 18346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:45.074967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:41995 (local address 127.0.0.1:56520)
0806 00:48:45.075631 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:45.075673 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:45.075819 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:45.076248 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:45.076260 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:45.077021 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:45.077160 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:45.077181 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.078715 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.078717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:45.079588 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:45.079605 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.080510 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.080513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:45.080635 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:45.081520 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:45.081543 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:45.083860 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:45.087311 (+ 3451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:45.087336 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:45.087354 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:45.087810 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:45.088381 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:45.088393 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:45.088395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:45.088578 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:45.089266 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:45.089272 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:45.089448 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 00:48:45.089605 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 00:48:45.089772 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":305,"threads_started":1}
I20260806 00:48:45.090791 17630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:45.075111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56520 (local address 127.15.147.190:41995)
0806 00:48:45.075333 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:45.075362 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:45.075779 (+ 417us) server_negotiation.cc:407] Connection header received
0806 00:48:45.075967 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:45.075971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:45.076066 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:45.076235 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:45.077354 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.078559 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.079748 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.080255 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.080527 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:45.084065 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:45.084079 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:45.084083 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:45.084124 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:45.087139 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:45.087953 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:45.087969 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:45.087971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:45.088130 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:45.088700 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:45.088704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:45.088706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:45.089116 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:45.089266 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:45.089720 (+ 454us) server_negotiation.cc:299] Negotiation successful
0806 00:48:45.089920 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":77}
I20260806 00:48:45.098520 17556 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:56520:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 00:48:45.100028 17556 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:48:45.110332 17954 tablet_service.cc:1467] Processing CreateTablet for tablet efd352ed345745339fc158ddf4e44398 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=efa4ee3b00504a4e8137d660a834900a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:48:45.110294 17745 tablet_service.cc:1467] Processing CreateTablet for tablet efd352ed345745339fc158ddf4e44398 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=efa4ee3b00504a4e8137d660a834900a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:48:45.111008 17954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efd352ed345745339fc158ddf4e44398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:45.111027 17745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efd352ed345745339fc158ddf4e44398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:45.115128 18300 tablet_bootstrap.cc:492] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb: Bootstrap starting.
I20260806 00:48:45.117017 18163 tablet_service.cc:1467] Processing CreateTablet for tablet efd352ed345745339fc158ddf4e44398 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=efa4ee3b00504a4e8137d660a834900a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:48:45.117308 18300 tablet_bootstrap.cc:654] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:45.117630 18163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efd352ed345745339fc158ddf4e44398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:45.118760 18299 tablet_bootstrap.cc:492] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962: Bootstrap starting.
I20260806 00:48:45.120440 18300 tablet_bootstrap.cc:492] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb: No bootstrap required, opened a new log
I20260806 00:48:45.120612 18300 ts_tablet_manager.cc:1397] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260806 00:48:45.120687 18299 tablet_bootstrap.cc:654] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:45.121515 18300 raft_consensus.cc:348] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } }
I20260806 00:48:45.123041 18300 raft_consensus.cc:374] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:45.123132 18300 raft_consensus.cc:729] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ac000c0683a4c3da82e3caf2b130adb, State: Initialized, Role: FOLLOWER
I20260806 00:48:45.123229 18299 tablet_bootstrap.cc:492] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962: No bootstrap required, opened a new log
I20260806 00:48:45.123345 18299 ts_tablet_manager.cc:1397] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 00:48:45.123373 18300 consensus_queue.cc:260] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } }
I20260806 00:48:45.123548 18298 tablet_bootstrap.cc:492] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc: Bootstrap starting.
I20260806 00:48:45.123919 18300 ts_tablet_manager.cc:1428] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 00:48:45.124115 18299 raft_consensus.cc:348] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } }
I20260806 00:48:45.124351 18299 raft_consensus.cc:374] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:45.124429 18299 raft_consensus.cc:729] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 834b9db8c8bb46c5ac148a84a8388962, State: Initialized, Role: FOLLOWER
I20260806 00:48:45.124661 18299 consensus_queue.cc:260] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } }
I20260806 00:48:45.125152 18299 ts_tablet_manager.cc:1428] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260806 00:48:45.125907 18298 tablet_bootstrap.cc:654] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:45.126784 18304 tablet_replica.cc:439] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } }
I20260806 00:48:45.127065 18304 tablet_replica.cc:442] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:48:45.128123 18298 tablet_bootstrap.cc:492] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc: No bootstrap required, opened a new log
I20260806 00:48:45.128329 18298 ts_tablet_manager.cc:1397] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260806 00:48:45.128190 18306 tablet_replica.cc:439] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } }
I20260806 00:48:45.128758 18306 tablet_replica.cc:442] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:48:45.129406 18298 raft_consensus.cc:348] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } }
I20260806 00:48:45.129617 18298 raft_consensus.cc:374] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:45.129679 18298 raft_consensus.cc:729] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5301f9aa1c144403a3e7adc74157dedc, State: Initialized, Role: FOLLOWER
I20260806 00:48:45.129882 18298 consensus_queue.cc:260] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } }
I20260806 00:48:45.130409 18298 ts_tablet_manager.cc:1428] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 00:48:45.131615 18305 tablet_replica.cc:439] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } }
I20260806 00:48:45.131873 18305 tablet_replica.cc:442] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:48:45.232216 18304 raft_consensus.cc:482] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:48:45.232555 18304 raft_consensus.cc:504] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } }
I20260806 00:48:45.234356 18304 leader_election.cc:288] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 834b9db8c8bb46c5ac148a84a8388962 (127.15.147.129:42667), 5301f9aa1c144403a3e7adc74157dedc (127.15.147.131:46637)
I20260806 00:48:45.243597 18295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:45.235266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:38899 (local address 127.15.147.131:46637)
0806 00:48:45.235525 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:45.235529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:45.236008 (+ 479us) server_negotiation.cc:407] Connection header received
0806 00:48:45.236415 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:45.236422 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:45.236539 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:45.236689 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:45.237334 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.238335 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.241582 (+ 3247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.242551 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.242790 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:45.242906 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:45.243232 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 00:48:45.243363 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 00:48:45.243629 18352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:45.234891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:46637 (local address 127.15.147.130:38899)
0806 00:48:45.235876 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:45.235903 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:45.236123 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:45.236895 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:45.236899 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:45.236932 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:45.237142 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:45.237150 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.238576 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.238588 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:45.241330 (+ 2742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:45.241352 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.242781 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.242788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:45.242963 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:45.243026 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 00:48:45.243181 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 00:48:45.243314 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"mutex_wait_us":750,"thread_start_us":144,"threads_started":1}
I20260806 00:48:45.244625 18189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efd352ed345745339fc158ddf4e44398" candidate_uuid: "9ac000c0683a4c3da82e3caf2b130adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5301f9aa1c144403a3e7adc74157dedc" is_pre_election: true
I20260806 00:48:45.244936 18189 raft_consensus.cc:2454] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ac000c0683a4c3da82e3caf2b130adb in term 0.
I20260806 00:48:45.245468 18289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:45.235462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:37887 (local address 127.15.147.129:42667)
0806 00:48:45.235690 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:45.235703 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:45.235932 (+ 229us) server_negotiation.cc:407] Connection header received
0806 00:48:45.236292 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:45.236328 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:45.236437 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:45.236589 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:45.237142 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.238308 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.243008 (+ 4700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.243973 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.244197 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:45.244348 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:45.245061 (+ 713us) server_negotiation.cc:299] Negotiation successful
0806 00:48:45.245227 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 00:48:45.245518 18351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:45.234881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:42667 (local address 127.15.147.130:37887)
0806 00:48:45.235755 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:45.235800 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:45.236023 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:45.236610 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:45.236618 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:45.236661 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:45.236942 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:45.236950 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.238576 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.238588 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:45.241330 (+ 2742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:45.241352 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.244540 (+ 3188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.244545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:45.244806 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:45.244874 (+ 68us) client_negotiation.cc:769] Sending connection context
0806 00:48:45.245030 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 00:48:45.245221 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"mutex_wait_us":287,"thread_start_us":142,"threads_started":1}
I20260806 00:48:45.246050 17887 leader_election.cc:302] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5301f9aa1c144403a3e7adc74157dedc, 9ac000c0683a4c3da82e3caf2b130adb; no voters:
I20260806 00:48:45.246582 18304 raft_consensus.cc:2781] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:48:45.246742 18304 raft_consensus.cc:482] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:48:45.246620 17783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efd352ed345745339fc158ddf4e44398" candidate_uuid: "9ac000c0683a4c3da82e3caf2b130adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "834b9db8c8bb46c5ac148a84a8388962" is_pre_election: true
I20260806 00:48:45.246856 18304 raft_consensus.cc:3037] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:45.247071 17783 raft_consensus.cc:2454] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ac000c0683a4c3da82e3caf2b130adb in term 0.
I20260806 00:48:45.249972 18304 raft_consensus.cc:504] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } }
I20260806 00:48:45.250716 18304 leader_election.cc:288] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [CANDIDATE]: Term 1 election: Requested vote from peers 834b9db8c8bb46c5ac148a84a8388962 (127.15.147.129:42667), 5301f9aa1c144403a3e7adc74157dedc (127.15.147.131:46637)
I20260806 00:48:45.251291 17783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efd352ed345745339fc158ddf4e44398" candidate_uuid: "9ac000c0683a4c3da82e3caf2b130adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "834b9db8c8bb46c5ac148a84a8388962"
I20260806 00:48:45.251329 18189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efd352ed345745339fc158ddf4e44398" candidate_uuid: "9ac000c0683a4c3da82e3caf2b130adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5301f9aa1c144403a3e7adc74157dedc"
I20260806 00:48:45.251472 17783 raft_consensus.cc:3037] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:45.251499 18189 raft_consensus.cc:3037] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:45.253665 17783 raft_consensus.cc:2454] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ac000c0683a4c3da82e3caf2b130adb in term 1.
I20260806 00:48:45.253665 18189 raft_consensus.cc:2454] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ac000c0683a4c3da82e3caf2b130adb in term 1.
I20260806 00:48:45.254123 17890 leader_election.cc:302] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 834b9db8c8bb46c5ac148a84a8388962, 9ac000c0683a4c3da82e3caf2b130adb; no voters:
I20260806 00:48:45.254400 18304 raft_consensus.cc:2781] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:45.254776 18304 raft_consensus.cc:686] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [term 1 LEADER]: Becoming Leader. State: Replica: 9ac000c0683a4c3da82e3caf2b130adb, State: Running, Role: LEADER
I20260806 00:48:45.255484 18304 consensus_queue.cc:237] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } }
I20260806 00:48:45.257907 18355 tablet_replica.cc:439] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb: TxnStatusTablet state changed. Reason: New leader 9ac000c0683a4c3da82e3caf2b130adb. Latest consensus state: current_term: 1 leader_uuid: "9ac000c0683a4c3da82e3caf2b130adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } }
I20260806 00:48:45.258136 18355 tablet_replica.cc:442] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:48:45.259945 18357 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 00:48:45.260239 18357 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 00:48:45.259910 17556 catalog_manager.cc:5495] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ac000c0683a4c3da82e3caf2b130adb (127.15.147.130). New cstate: current_term: 1 leader_uuid: "9ac000c0683a4c3da82e3caf2b130adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } health_report { overall_health: UNKNOWN } } }
I20260806 00:48:45.297842 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:45.289581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35900 (local address 127.15.147.130:44733)
0806 00:48:45.289810 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:45.289825 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:45.290331 (+ 506us) server_negotiation.cc:407] Connection header received
0806 00:48:45.290455 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:45.290459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:45.290590 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:45.290770 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:45.291228 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.292747 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.294665 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.295263 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.295508 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:45.295994 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:45.297041 (+ 1047us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:45.297229 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:45.297398 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 00:48:45.297512 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260806 00:48:45.297828 18346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:45.289700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:44733 (local address 127.0.0.1:35900)
0806 00:48:45.289987 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:45.290014 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:45.290178 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:45.290821 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:45.290825 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:45.290834 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:48:45.291021 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:45.291039 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.292903 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.292907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:45.294447 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:45.294461 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.295503 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.295508 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:45.295683 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:45.295750 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:48:45.297208 (+ 1458us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:48:45.297223 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 00:48:45.297363 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 00:48:45.297514 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260806 00:48:45.303465 18189 raft_consensus.cc:1261] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc [term 1 FOLLOWER]: Refusing update from remote peer 9ac000c0683a4c3da82e3caf2b130adb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:45.305172 17783 raft_consensus.cc:1261] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962 [term 1 FOLLOWER]: Refusing update from remote peer 9ac000c0683a4c3da82e3caf2b130adb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:45.306038 18304 consensus_queue.cc:1035] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:45.306974 18304 consensus_queue.cc:1035] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:45.309733 18306 tablet_replica.cc:439] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962: TxnStatusTablet state changed. Reason: New leader 9ac000c0683a4c3da82e3caf2b130adb. Latest consensus state: current_term: 1 leader_uuid: "9ac000c0683a4c3da82e3caf2b130adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } }
I20260806 00:48:45.309921 18306 tablet_replica.cc:442] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:48:45.310225 18305 tablet_replica.cc:439] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc: TxnStatusTablet state changed. Reason: New leader 9ac000c0683a4c3da82e3caf2b130adb. Latest consensus state: current_term: 1 leader_uuid: "9ac000c0683a4c3da82e3caf2b130adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } }
I20260806 00:48:45.310443 18305 tablet_replica.cc:442] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:48:45.312893 18355 tablet_replica.cc:439] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9ac000c0683a4c3da82e3caf2b130adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } }
I20260806 00:48:45.313074 18355 tablet_replica.cc:442] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:48:45.319365 18304 tablet_replica.cc:439] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9ac000c0683a4c3da82e3caf2b130adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } }
I20260806 00:48:45.319607 18304 tablet_replica.cc:442] T efd352ed345745339fc158ddf4e44398 P 9ac000c0683a4c3da82e3caf2b130adb: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:48:45.321611 18306 tablet_replica.cc:439] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9ac000c0683a4c3da82e3caf2b130adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } }
I20260806 00:48:45.321823 18306 tablet_replica.cc:442] T efd352ed345745339fc158ddf4e44398 P 834b9db8c8bb46c5ac148a84a8388962: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:48:45.322832 18317 mvcc.cc:205] Tried to move back new op lower bound from 7315363124434063360 to 7315363124250947584. Current Snapshot: MvccSnapshot[applied={T|T < 7315363124434063360 or (T in {7315363124434063360})}]
I20260806 00:48:45.323200 18305 tablet_replica.cc:439] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9ac000c0683a4c3da82e3caf2b130adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834b9db8c8bb46c5ac148a84a8388962" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42667 } } peers { permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 } } peers { permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 } } }
I20260806 00:48:45.323437 18305 tablet_replica.cc:442] T efd352ed345745339fc158ddf4e44398 P 5301f9aa1c144403a3e7adc74157dedc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:48:45.349728 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:45.343328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:44733 (local address 127.15.147.129:45843)
0806 00:48:45.344051 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:45.344073 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:45.344231 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:45.344832 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:45.344834 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:45.344842 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:48:45.345006 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:45.345012 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.346611 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.346615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:45.347536 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:45.347554 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.348566 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.348569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:45.348665 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:45.348741 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:48:45.349311 (+ 570us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:48:45.349320 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 00:48:45.349417 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 00:48:45.349557 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":266,"threads_started":1}
I20260806 00:48:45.350421 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:45.343433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:45843 (local address 127.15.147.130:44733)
0806 00:48:45.343622 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:45.343633 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:45.344198 (+ 565us) server_negotiation.cc:407] Connection header received
0806 00:48:45.344503 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:45.344518 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:45.344633 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:45.344820 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:45.345172 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.346422 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.347707 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.348413 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.348566 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:45.348995 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:45.349145 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:45.349299 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:45.349467 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 00:48:45.349594 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 00:48:45.362810 18189 raft_consensus.cc:1261] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc [term 1 FOLLOWER]: Refusing update from remote peer 834b9db8c8bb46c5ac148a84a8388962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:45.362857 17981 raft_consensus.cc:1261] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb [term 1 FOLLOWER]: Refusing update from remote peer 834b9db8c8bb46c5ac148a84a8388962: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:45.363713 18328 consensus_queue.cc:1035] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5301f9aa1c144403a3e7adc74157dedc" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:45.364029 18306 consensus_queue.cc:1035] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ac000c0683a4c3da82e3caf2b130adb" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:45.370249 18313 mvcc.cc:205] Tried to move back new op lower bound from 7315363124680278016 to 7315363123343081472. Current Snapshot: MvccSnapshot[applied={T|T < 7315363124680278016}]
I20260806 00:48:45.411615 18289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:45.403841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:59531 (local address 127.15.147.129:42667)
0806 00:48:45.404106 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:45.404111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:45.405081 (+ 970us) server_negotiation.cc:407] Connection header received
0806 00:48:45.405408 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:45.405412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:45.405491 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:45.405644 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:45.406689 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.407456 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.409359 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.409832 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.409942 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:45.410717 (+ 775us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:45.410829 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:45.410992 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:45.411270 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 00:48:45.411384 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 00:48:45.411868 18375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:45.403855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:42667 (local address 127.15.147.130:59531)
0806 00:48:45.404884 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:45.404927 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:45.405200 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:45.405977 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:45.405979 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:45.405986 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:48:45.406410 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:45.406421 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.407646 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.407649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:45.409057 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:45.409076 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.410097 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.410101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:45.410368 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:45.410476 (+ 108us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:48:45.411067 (+ 591us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:48:45.411074 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 00:48:45.411218 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 00:48:45.411365 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"thread_start_us":240,"threads_started":1}
I20260806 00:48:45.454128 17556 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:48:45.454329 17556 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:48:45.457507 17556 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 85d314529f6e4b13bbedf99781761f65: Sending DeleteTablet for 3 replicas of tablet 5388334122e84c5ca7d35c8b087fcc66
I20260806 00:48:45.458578 17954 tablet_service.cc:1514] Processing DeleteTablet for tablet 5388334122e84c5ca7d35c8b087fcc66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:48:45 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:35886
I20260806 00:48:45.458578 17745 tablet_service.cc:1514] Processing DeleteTablet for tablet 5388334122e84c5ca7d35c8b087fcc66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:48:45 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:52700
I20260806 00:48:45.458731 18163 tablet_service.cc:1514] Processing DeleteTablet for tablet 5388334122e84c5ca7d35c8b087fcc66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:48:45 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:55914
I20260806 00:48:45.459004 18334 tablet_replica.cc:330] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb: stopping tablet replica
I20260806 00:48:45.459028 18333 tablet_replica.cc:330] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc: stopping tablet replica
I20260806 00:48:45.459218 18334 raft_consensus.cc:2229] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:48:45.459237 18333 raft_consensus.cc:2229] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:48:45.459403 18334 raft_consensus.cc:2258] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:48:45.459488 18333 raft_consensus.cc:2258] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:48:45.459751 18335 tablet_replica.cc:330] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962: stopping tablet replica
I20260806 00:48:45.459962 18335 raft_consensus.cc:2229] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:48:45.460369 18335 raft_consensus.cc:2258] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:48:45.461058 18333 ts_tablet_manager.cc:1905] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:48:45.462432 18335 ts_tablet_manager.cc:1905] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:48:45.465725 18333 ts_tablet_manager.cc:1918] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:48:45.465822 18333 log.cc:1196] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/5388334122e84c5ca7d35c8b087fcc66
I20260806 00:48:45.466236 18333 ts_tablet_manager.cc:1939] T 5388334122e84c5ca7d35c8b087fcc66 P 5301f9aa1c144403a3e7adc74157dedc: Deleting consensus metadata
I20260806 00:48:45.467264 17537 catalog_manager.cc:4841] TS 5301f9aa1c144403a3e7adc74157dedc (127.15.147.131:46637): tablet 5388334122e84c5ca7d35c8b087fcc66 (table test-table [id=a7597e9da7c64ae6b4b103e5e0e895f1]) successfully deleted
I20260806 00:48:45.467386 18335 ts_tablet_manager.cc:1918] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:48:45.467512 18335 log.cc:1196] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-0/wal/wals/5388334122e84c5ca7d35c8b087fcc66
I20260806 00:48:45.468030 18335 ts_tablet_manager.cc:1939] T 5388334122e84c5ca7d35c8b087fcc66 P 834b9db8c8bb46c5ac148a84a8388962: Deleting consensus metadata
I20260806 00:48:45.469604 17540 catalog_manager.cc:4841] TS 834b9db8c8bb46c5ac148a84a8388962 (127.15.147.129:42667): tablet 5388334122e84c5ca7d35c8b087fcc66 (table test-table [id=a7597e9da7c64ae6b4b103e5e0e895f1]) successfully deleted
I20260806 00:48:45.470631 18334 ts_tablet_manager.cc:1905] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:48:45.475158 18334 ts_tablet_manager.cc:1918] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:48:45.475268 18334 log.cc:1196] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/5388334122e84c5ca7d35c8b087fcc66
I20260806 00:48:45.475744 18334 ts_tablet_manager.cc:1939] T 5388334122e84c5ca7d35c8b087fcc66 P 9ac000c0683a4c3da82e3caf2b130adb: Deleting consensus metadata
I20260806 00:48:45.477155 17536 catalog_manager.cc:4841] TS 9ac000c0683a4c3da82e3caf2b130adb (127.15.147.130:44733): tablet 5388334122e84c5ca7d35c8b087fcc66 (table test-table [id=a7597e9da7c64ae6b4b103e5e0e895f1]) successfully deleted
Aug 06 00:48:45 dist-test-slave-45dx krb5kdc[17496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977324, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.147.129@KRBTEST.COM
I20260806 00:48:45.486094 18289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:45.468604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52714 (local address 127.15.147.129:42667)
0806 00:48:45.468831 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:45.468842 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:45.468856 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:48:45.468911 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:45.468913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:45.468982 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:45.469140 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:45.470696 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.471844 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.473758 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.474325 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.474592 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:45.479656 (+ 5064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:45.479678 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:45.479707 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:45.479749 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:45.483208 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:45.483859 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:45.483866 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:45.483868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:45.484026 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:45.484473 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:45.484485 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:45.484488 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:45.485017 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:45.485411 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:45.485600 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 00:48:45.485813 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
W20260806 00:48:45.486841 17812 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52714
I20260806 00:48:45.503150 17630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:45.489549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56534 (local address 127.15.147.190:41995)
0806 00:48:45.489746 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:45.489752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:45.489986 (+ 234us) server_negotiation.cc:407] Connection header received
0806 00:48:45.490244 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:45.490248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:45.490303 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:45.490454 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:45.491424 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.492133 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.493355 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:45.493862 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:45.494048 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:45.497738 (+ 3690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:45.497743 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:45.497745 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:45.497783 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:45.500105 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:45.501212 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:45.501215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:45.501216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:45.501330 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:45.501861 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:45.501863 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:45.501864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:45.502144 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:45.502294 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:45.502818 (+ 524us) server_negotiation.cc:299] Negotiation successful
0806 00:48:45.502967 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 00:48:45.504037 17556 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56534
I20260806 00:48:45.506120 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 17660
I20260806 00:48:45.534722 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 17860
I20260806 00:48:45.554145 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 18065
I20260806 00:48:45.569326 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 17514
2026-08-06T00:48:45Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3144 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:45 dist-test-slave-45dx krb5kdc[18385](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:45 dist-test-slave-45dx krb5kdc[18385](info): set up 2 sockets
Aug 06 00:48:45 dist-test-slave-45dx krb5kdc[18385](info): commencing operation
krb5kdc: starting...
W20260806 00:48:46.896404 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.000s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:48:46 dist-test-slave-45dx krb5kdc[18385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:48:47.025401 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:33337
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:40349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:33337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:47.140635 18404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:47.140877 18404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:47.140913 18404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:47.144806 18404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:47.144852 18404 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 00:48:47.144868 18404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:47.144886 18404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:47.144901 18404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:47.144915 18404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:47.150429 18404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:33337
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:33337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:47.151324 18404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:47.152221 18404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:47.160698 18416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:47.160778 18414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:47.160869 18404 server_base.cc:1034] running on GCE node
W20260806 00:48:47.160671 18413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:47.161501 18404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:47.162550 18404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:47.163745 18404 hybrid_clock.cc:648] HybridClock initialized: now 1785977327163726 us; error 35 us; skew 500 ppm
Aug 06 00:48:47 dist-test-slave-45dx krb5kdc[18385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977327, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:47.166602 18404 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:48:47.167500 18404 webserver.cc:460] Webserver started at http://127.15.147.190:40593/ using document root <none> and password file <none>
I20260806 00:48:47.168067 18404 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:47.168165 18404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:47.168457 18404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:47.170485 18404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "e0729b77006743698f3584e1fea9a1c0"
format_stamp: "Formatted at 2026-08-06 00:48:47 on dist-test-slave-45dx"
server_key: "9f5078df63a22422438a5e83dfe76c7a"
server_key_iv: "ced7461b4021591e0d4a17a4ebf11a2d"
server_key_version: "encryptionkey@0"
I20260806 00:48:47.171089 18404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "e0729b77006743698f3584e1fea9a1c0"
format_stamp: "Formatted at 2026-08-06 00:48:47 on dist-test-slave-45dx"
server_key: "9f5078df63a22422438a5e83dfe76c7a"
server_key_iv: "ced7461b4021591e0d4a17a4ebf11a2d"
server_key_version: "encryptionkey@0"
I20260806 00:48:47.174816 18404 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:48:47.177124 18423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:47.178277 18404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:48:47.178424 18404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "e0729b77006743698f3584e1fea9a1c0"
format_stamp: "Formatted at 2026-08-06 00:48:47 on dist-test-slave-45dx"
server_key: "9f5078df63a22422438a5e83dfe76c7a"
server_key_iv: "ced7461b4021591e0d4a17a4ebf11a2d"
server_key_version: "encryptionkey@0"
I20260806 00:48:47.178534 18404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:47.189319 18404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:47.192116 18404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:47.192348 18404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:47.200702 18404 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:33337
I20260806 00:48:47.200728 18525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:33337 every 8 connection(s)
I20260806 00:48:47.201644 18404 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:47.201967 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 18404
I20260806 00:48:47.202049 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:48:47.202384 15950 external_mini_cluster.cc:1428] Setting key b57a52f549880e0869a074a9f5cd4650
I20260806 00:48:47.205039 18526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:47.211571 18526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0: Bootstrap starting.
Aug 06 00:48:47 dist-test-slave-45dx krb5kdc[18385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:47.214138 18526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:47.215301 18526 log.cc:824] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:47.217337 18526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0: No bootstrap required, opened a new log
I20260806 00:48:47.217600 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.203837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38382 (local address 127.15.147.190:33337)
0806 00:48:47.204287 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:47.204338 (+ 51us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:47.204382 (+ 44us) server_negotiation.cc:407] Connection header received
0806 00:48:47.205287 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:47.205322 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:47.205745 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:47.206098 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:47.206802 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.208228 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.209160 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.209827 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.210008 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.212754 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:47.212760 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:47.212771 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:47.212803 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:47.214581 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:47.215034 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:47.215039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:47.215043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:47.215158 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:47.215490 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:47.215493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:47.215494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:47.216023 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:47.216218 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:47.216550 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 00:48:47.216842 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":127,"threads_started":1}
I20260806 00:48:47.220862 18526 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0729b77006743698f3584e1fea9a1c0" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33337 } }
I20260806 00:48:47.221128 18526 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:47.221191 18526 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0729b77006743698f3584e1fea9a1c0, State: Initialized, Role: FOLLOWER
I20260806 00:48:47.221669 18526 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0729b77006743698f3584e1fea9a1c0" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33337 } }
I20260806 00:48:47.221849 18526 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:47.221920 18526 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:47.222050 18526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:47.222939 18526 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0729b77006743698f3584e1fea9a1c0" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33337 } }
I20260806 00:48:47.223266 18526 leader_election.cc:302] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0729b77006743698f3584e1fea9a1c0; no voters:
I20260806 00:48:47.223608 18526 leader_election.cc:288] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:47.223732 18537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:47.223989 18537 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0 [term 1 LEADER]: Becoming Leader. State: Replica: e0729b77006743698f3584e1fea9a1c0, State: Running, Role: LEADER
I20260806 00:48:47.224293 18537 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0729b77006743698f3584e1fea9a1c0" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33337 } }
I20260806 00:48:47.224879 18526 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:47.226073 18538 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0729b77006743698f3584e1fea9a1c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0729b77006743698f3584e1fea9a1c0" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33337 } } }
I20260806 00:48:47.226210 18538 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:47.226498 18539 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0729b77006743698f3584e1fea9a1c0. Latest consensus state: current_term: 1 leader_uuid: "e0729b77006743698f3584e1fea9a1c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0729b77006743698f3584e1fea9a1c0" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33337 } } }
I20260806 00:48:47.226614 18539 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:47.226895 18551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:47.229672 18551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:47.234727 18551 catalog_manager.cc:1347] Generated new cluster ID: 50c1d5eb6de043108552b2f52a4f3127
I20260806 00:48:47.234789 18551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:47.252714 18551 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:47.253608 18551 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:47.258513 18551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0: Generated new TSK 0
I20260806 00:48:47.259171 18551 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
I20260806 00:48:47.295449 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:33337
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:40349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:47.408490 18569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:47.408758 18569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:47.408844 18569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:47.413009 18569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:47.413072 18569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:47.413136 18569 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 00:48:47.413166 18569 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:48:47.413218 18569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:48:47.418603 18569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.147.190:33337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:47.419780 18569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:47.420912 18569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:47.431211 18579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:47.431330 18569 server_base.cc:1034] running on GCE node
W20260806 00:48:47.431212 18580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:47.431272 18583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:47.431931 18569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:47.432653 18569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:47.433846 18569 hybrid_clock.cc:648] HybridClock initialized: now 1785977327433829 us; error 26 us; skew 500 ppm
Aug 06 00:48:47 dist-test-slave-45dx krb5kdc[18385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977327, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:47.441001 18569 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:48:47.441931 18569 webserver.cc:460] Webserver started at http://127.15.147.129:33683/ using document root <none> and password file <none>
I20260806 00:48:47.442497 18569 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:47.442592 18569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:47.442826 18569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:47.444931 18569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "13b69f2f7f684c4bae51d211aee4ac9c"
format_stamp: "Formatted at 2026-08-06 00:48:47 on dist-test-slave-45dx"
server_key: "1d18f568632fd80c41433840fef84b82"
server_key_iv: "1e257ddf548e90f39659cfea4f21f1b3"
server_key_version: "encryptionkey@0"
I20260806 00:48:47.445504 18569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "13b69f2f7f684c4bae51d211aee4ac9c"
format_stamp: "Formatted at 2026-08-06 00:48:47 on dist-test-slave-45dx"
server_key: "1d18f568632fd80c41433840fef84b82"
server_key_iv: "1e257ddf548e90f39659cfea4f21f1b3"
server_key_version: "encryptionkey@0"
I20260806 00:48:47.449163 18569 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:48:47.451355 18595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:47.452678 18569 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:48:47.452812 18569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "13b69f2f7f684c4bae51d211aee4ac9c"
format_stamp: "Formatted at 2026-08-06 00:48:47 on dist-test-slave-45dx"
server_key: "1d18f568632fd80c41433840fef84b82"
server_key_iv: "1e257ddf548e90f39659cfea4f21f1b3"
server_key_version: "encryptionkey@0"
I20260806 00:48:47.452955 18569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:47.466810 18569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:47.470238 18569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:47.470453 18569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:47.472012 18569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:48:47.472082 18569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:47.472173 18569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:48:47.472198 18569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:48:47 dist-test-slave-45dx krb5kdc[18385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977327, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:47.483984 18569 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:40491
I20260806 00:48:47.484450 18792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:40491 every 8 connection(s)
I20260806 00:48:47.485003 18569 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:48:47.487299 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.472706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:59617 (local address 127.15.147.190:33337)
0806 00:48:47.472830 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:47.472841 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:47.474532 (+ 1691us) server_negotiation.cc:407] Connection header received
0806 00:48:47.475770 (+ 1238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:47.475773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:47.475831 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:47.475924 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:47.477563 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.478427 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.479811 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.480067 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.480191 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.483365 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:47.483368 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:47.483370 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:47.483405 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:47.485085 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:47.485748 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:47.485753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:47.485756 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:47.485866 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:47.486229 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:47.486234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:47.486235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:47.486424 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:47.486536 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:47.487001 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 00:48:47.487129 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 00:48:47.488018 18607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.473812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:33337 (local address 127.15.147.129:59617)
0806 00:48:47.474328 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:47.474402 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:47.475567 (+ 1165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:47.476118 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:47.476127 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:47.476639 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:47.477375 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:47.477399 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.478562 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.478566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:47.479673 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:47.479684 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.480254 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.480258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:47.480473 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.481110 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:47.481132 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:47.483200 (+ 2068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:47.485241 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:47.485247 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:47.485259 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:47.485616 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:47.485978 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:47.485982 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:47.485984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:47.486126 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:47.486535 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:47.486542 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:47.486774 (+ 232us) client_negotiation.cc:769] Sending connection context
0806 00:48:47.486953 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 00:48:47.487247 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":127,"threads_started":1}
I20260806 00:48:47.489430 18794 heartbeater.cc:347] Connected to a master server at 127.15.147.190:33337
I20260806 00:48:47.489588 18794 heartbeater.cc:464] Registering TS with master...
I20260806 00:48:47.490084 18794 heartbeater.cc:511] Master 127.15.147.190:33337 requested a full tablet report, sending...
I20260806 00:48:47.491600 18450 ts_manager.cc:194] Registered new tserver with Master: 13b69f2f7f684c4bae51d211aee4ac9c (127.15.147.129:40491)
I20260806 00:48:47.492575 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 18569
I20260806 00:48:47.492702 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:48:47.492854 18450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:59617
I20260806 00:48:47.493045 15950 external_mini_cluster.cc:1428] Setting key 3732df424905f2266b69126ad4d261a8
I20260806 00:48:47.500485 18804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.491474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:33337 (local address 127.15.147.129:58117)
0806 00:48:47.491741 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:47.491755 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:47.491834 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:47.493734 (+ 1900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:47.493738 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:47.493945 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:47.494067 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:47.494073 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.495064 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.495067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:47.495619 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:47.495627 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.496390 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.496393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:47.496465 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.496983 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:47.497001 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:47.497315 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:47.498908 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:47.498911 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:47.498912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:47.499123 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:47.499410 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:47.499413 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:47.499414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:47.499503 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:47.499878 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:47.499881 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:47.499981 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 00:48:47.500088 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 00:48:47.500274 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":106,"threads_started":1}
I20260806 00:48:47.500561 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.491564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:58117 (local address 127.15.147.190:33337)
0806 00:48:47.493490 (+ 1926us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:47.493497 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:47.493511 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:48:47.493547 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:47.493550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:47.493609 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:47.493688 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:47.494205 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.494900 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.495748 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.496206 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.496370 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.497452 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:47.497462 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:47.497464 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:47.497485 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:47.498788 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:47.499231 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:47.499239 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:47.499240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:47.499315 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:47.499603 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:47.499607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:47.499608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:47.499777 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:47.499884 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:47.500295 (+ 411us) server_negotiation.cc:299] Negotiation successful
0806 00:48:47.500425 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1855}
WARNING: no policy specified for kudu/127.15.147.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
I20260806 00:48:47.522488 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:33337
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:40349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:47.636878 18808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:47.637118 18808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:47.637202 18808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:47.641077 18808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:47.641129 18808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:47.641165 18808 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 00:48:47.641218 18808 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:48:47.641286 18808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:48:47.646829 18808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.147.190:33337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:47.647934 18808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:47.649019 18808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:47.658943 18817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:47.658919 18814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:47.658921 18815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:47.658991 18808 server_base.cc:1034] running on GCE node
I20260806 00:48:47.659672 18808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:47.660238 18808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:47.661407 18808 hybrid_clock.cc:648] HybridClock initialized: now 1785977327661378 us; error 43 us; skew 500 ppm
Aug 06 00:48:47 dist-test-slave-45dx krb5kdc[18385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977327, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:47.664106 18808 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:48:47.665011 18808 webserver.cc:460] Webserver started at http://127.15.147.130:41415/ using document root <none> and password file <none>
I20260806 00:48:47.665577 18808 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:47.665670 18808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:47.665905 18808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:47.667932 18808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "4a5b0afc8c84409981faf15563753388"
format_stamp: "Formatted at 2026-08-06 00:48:47 on dist-test-slave-45dx"
server_key: "652b2039cf53f64fcb1ea77fcb966913"
server_key_iv: "41d91214235d62c5bd0d63088b16e136"
server_key_version: "encryptionkey@0"
I20260806 00:48:47.668625 18808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "4a5b0afc8c84409981faf15563753388"
format_stamp: "Formatted at 2026-08-06 00:48:47 on dist-test-slave-45dx"
server_key: "652b2039cf53f64fcb1ea77fcb966913"
server_key_iv: "41d91214235d62c5bd0d63088b16e136"
server_key_version: "encryptionkey@0"
I20260806 00:48:47.672281 18808 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:48:47.674646 18826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:47.675822 18808 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:48:47.675967 18808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "4a5b0afc8c84409981faf15563753388"
format_stamp: "Formatted at 2026-08-06 00:48:47 on dist-test-slave-45dx"
server_key: "652b2039cf53f64fcb1ea77fcb966913"
server_key_iv: "41d91214235d62c5bd0d63088b16e136"
server_key_version: "encryptionkey@0"
I20260806 00:48:47.676082 18808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:47.689842 18808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:47.692715 18808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:47.692942 18808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:47.694396 18808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:48:47.694458 18808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:47.694521 18808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:48:47.694556 18808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:48:47 dist-test-slave-45dx krb5kdc[18385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977327, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:47.705960 18808 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:46793
I20260806 00:48:47.706012 19019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:46793 every 8 connection(s)
I20260806 00:48:47.707099 18808 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:48:47.709193 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 18808
I20260806 00:48:47.709311 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:48:47.709641 15950 external_mini_cluster.cc:1428] Setting key 4f010a13e579dc65e1348d55e1bc4339
I20260806 00:48:47.709781 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.695243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:46583 (local address 127.15.147.190:33337)
0806 00:48:47.695369 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:47.695373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:47.696512 (+ 1139us) server_negotiation.cc:407] Connection header received
0806 00:48:47.697507 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:47.697512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:47.697582 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:47.697681 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:47.699479 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.700404 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.702371 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.702752 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.702872 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.705904 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:47.705908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:47.705911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:47.705941 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:47.707450 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:47.708095 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:47.708099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:47.708101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:47.708198 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:47.708610 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:47.708613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:47.708614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:47.708791 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:47.708870 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:47.709461 (+ 591us) server_negotiation.cc:299] Negotiation successful
0806 00:48:47.709597 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 00:48:47.710570 18838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.695775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:33337 (local address 127.15.147.130:46583)
0806 00:48:47.696368 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:47.696411 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:47.697258 (+ 847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:47.698014 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:47.698024 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:47.698579 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:47.699295 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:47.699318 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.700667 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.700674 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:47.702226 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:47.702242 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.702923 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.702930 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:47.703139 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.703796 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:47.703825 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:47.705707 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:47.707584 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:47.707590 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:47.707603 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:47.707975 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:47.708293 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:47.708295 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:47.708296 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:47.708500 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:47.708912 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:47.708917 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:47.709143 (+ 226us) client_negotiation.cc:769] Sending connection context
0806 00:48:47.709330 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 00:48:47.709622 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":146,"threads_started":1}
I20260806 00:48:47.712045 19020 heartbeater.cc:347] Connected to a master server at 127.15.147.190:33337
I20260806 00:48:47.712195 19020 heartbeater.cc:464] Registering TS with master...
I20260806 00:48:47.712800 19020 heartbeater.cc:511] Master 127.15.147.190:33337 requested a full tablet report, sending...
I20260806 00:48:47.713948 18451 ts_manager.cc:194] Registered new tserver with Master: 4a5b0afc8c84409981faf15563753388 (127.15.147.130:46793)
I20260806 00:48:47.714751 18451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:46583
WARNING: no policy specified for kudu/127.15.147.131@KRBTEST.COM; defaulting to no policy
I20260806 00:48:47.722477 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.714358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:52371 (local address 127.15.147.190:33337)
0806 00:48:47.714492 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:47.714496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:47.714766 (+ 270us) server_negotiation.cc:407] Connection header received
0806 00:48:47.714974 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:47.714977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:47.715028 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:47.715101 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:47.715741 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.716579 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.717621 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.718050 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.718178 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.719174 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:47.719180 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:47.719185 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:47.719211 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:47.720831 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:47.721233 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:47.721237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:47.721238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:47.721322 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:47.721594 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:47.721599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:47.721601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:47.721838 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:47.721970 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:47.722233 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 00:48:47.722340 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:48:47.722494 19034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.714311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:33337 (local address 127.15.147.130:52371)
0806 00:48:47.714680 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:47.714691 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:47.714805 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:47.715215 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:47.715218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:47.715441 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:47.715605 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:47.715614 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.716728 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.716731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:47.717499 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:47.717509 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.718168 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.718170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:47.718235 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.718733 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:47.718749 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:47.719033 (+ 284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:47.720949 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:47.720951 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:47.720953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:47.721134 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:47.721419 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:47.721421 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:47.721422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:47.721494 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:47.721960 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:47.721963 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:47.722059 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 00:48:47.722167 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 00:48:47.722305 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":2,"thread_start_us":110,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
I20260806 00:48:47.739706 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:33337
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:40349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:47.851181 19038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:47.851418 19038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:47.851459 19038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:47.855484 19038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:47.855543 19038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:47.855578 19038 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 00:48:47.855590 19038 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:48:47.855618 19038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:48:47.860812 19038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.147.190:33337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:47.861721 19038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:47.862671 19038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:47.872042 19043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:47.872057 19050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:47.872047 19045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:47.872457 19038 server_base.cc:1034] running on GCE node
I20260806 00:48:47.872881 19038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:47.873514 19038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:47.874679 19038 hybrid_clock.cc:648] HybridClock initialized: now 1785977327874663 us; error 27 us; skew 500 ppm
Aug 06 00:48:47 dist-test-slave-45dx krb5kdc[18385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977327, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:47.877511 19038 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:48:47.878319 19038 webserver.cc:460] Webserver started at http://127.15.147.131:41881/ using document root <none> and password file <none>
I20260806 00:48:47.878913 19038 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:47.878981 19038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:47.879155 19038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:47.881227 19038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "944d83c0514b408482a985429baa67c4"
format_stamp: "Formatted at 2026-08-06 00:48:47 on dist-test-slave-45dx"
server_key: "3d6ec0b7f005409c59c9e720b95213f8"
server_key_iv: "4c962f21373252f5e92c16c2117a0e91"
server_key_version: "encryptionkey@0"
I20260806 00:48:47.881711 19038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "944d83c0514b408482a985429baa67c4"
format_stamp: "Formatted at 2026-08-06 00:48:47 on dist-test-slave-45dx"
server_key: "3d6ec0b7f005409c59c9e720b95213f8"
server_key_iv: "4c962f21373252f5e92c16c2117a0e91"
server_key_version: "encryptionkey@0"
I20260806 00:48:47.885284 19038 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 00:48:47.887547 19063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:47.888834 19038 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:48:47.888949 19038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "944d83c0514b408482a985429baa67c4"
format_stamp: "Formatted at 2026-08-06 00:48:47 on dist-test-slave-45dx"
server_key: "3d6ec0b7f005409c59c9e720b95213f8"
server_key_iv: "4c962f21373252f5e92c16c2117a0e91"
server_key_version: "encryptionkey@0"
I20260806 00:48:47.889076 19038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:47.907573 19038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:47.910485 19038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:47.910634 19038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:47.912122 19038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:48:47.912206 19038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:47.912300 19038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:48:47.912381 19038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 06 00:48:47 dist-test-slave-45dx krb5kdc[18385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977327, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:47.924273 19038 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:44909
I20260806 00:48:47.925002 19226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:44909 every 8 connection(s)
I20260806 00:48:47.925320 19038 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:48:47.927168 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.913106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:46857 (local address 127.15.147.190:33337)
0806 00:48:47.913243 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:47.913249 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:47.914457 (+ 1208us) server_negotiation.cc:407] Connection header received
0806 00:48:47.915733 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:47.915738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:47.915807 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:47.915871 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:47.917732 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.918516 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.919652 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.919900 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.920016 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.923237 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:47.923240 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:47.923243 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:47.923268 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:47.924879 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:47.925532 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:47.925536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:47.925538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:47.925639 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:47.926100 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:47.926103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:47.926105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:47.926336 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:47.926438 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:47.926952 (+ 514us) server_negotiation.cc:299] Negotiation successful
0806 00:48:47.927052 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:48:47.927886 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 19038
I20260806 00:48:47.928022 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:48:47.928110 19074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.913650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:33337 (local address 127.15.147.131:46857)
0806 00:48:47.914278 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:47.914340 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:47.915483 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:47.916278 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:47.916288 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:47.916820 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:47.917525 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:47.917549 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.918647 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.918650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:47.919517 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:47.919527 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.920064 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.920068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:47.920274 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.921098 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:47.921114 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:47.923052 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:47.924997 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:47.925002 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:47.925014 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:47.925369 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:47.925776 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:47.925780 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:47.925782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:47.925979 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:47.926454 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:47.926459 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:47.926710 (+ 251us) client_negotiation.cc:769] Sending connection context
0806 00:48:47.926921 (+ 211us) client_negotiation.cc:240] Negotiation successful
0806 00:48:47.927167 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":138,"threads_started":1}
I20260806 00:48:47.928397 15950 external_mini_cluster.cc:1428] Setting key 1744ea9dda2f6ab673e3cd0a937839d2
I20260806 00:48:47.930440 19227 heartbeater.cc:347] Connected to a master server at 127.15.147.190:33337
I20260806 00:48:47.930599 19227 heartbeater.cc:464] Registering TS with master...
I20260806 00:48:47.931185 19227 heartbeater.cc:511] Master 127.15.147.190:33337 requested a full tablet report, sending...
I20260806 00:48:47.932255 18451 ts_manager.cc:194] Registered new tserver with Master: 944d83c0514b408482a985429baa67c4 (127.15.147.131:44909)
I20260806 00:48:47.932868 18451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:46857
I20260806 00:48:47.939284 19237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.933174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:33337 (local address 127.15.147.131:55527)
0806 00:48:47.933434 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:47.933449 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:47.933550 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:47.933774 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:47.933777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:47.933988 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:47.934114 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:47.934122 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.934833 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.934836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:47.935410 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:47.935419 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.935807 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.935809 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:47.935877 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.936275 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:47.936286 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:47.936649 (+ 363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:47.938078 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:47.938081 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:47.938082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:47.938266 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:47.938502 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:47.938504 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:47.938505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:47.938578 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:47.938919 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:47.938921 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:47.938990 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 00:48:47.939067 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 00:48:47.939164 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":96,"threads_started":1}
I20260806 00:48:47.939306 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.932153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:55527 (local address 127.15.147.190:33337)
0806 00:48:47.932275 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:47.932278 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:47.933526 (+ 1248us) server_negotiation.cc:407] Connection header received
0806 00:48:47.933628 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:47.933631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:47.933667 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:47.933730 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:47.934232 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.934705 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.935509 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.935714 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.935783 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.936772 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:47.936775 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:47.936777 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:47.936798 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:47.937985 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:47.938353 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:47.938355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:47.938356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:47.938425 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:47.938671 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:47.938673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:47.938674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:47.938820 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:47.938903 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:47.939124 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 00:48:47.939207 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:48:47.942334 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 00:48:47 dist-test-slave-45dx krb5kdc[18385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 00:48:47 dist-test-slave-45dx krb5kdc[18385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:47.970306 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.961942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38390 (local address 127.15.147.190:33337)
0806 00:48:47.962085 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:47.962089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:47.962103 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:48:47.962144 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:47.962146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:47.962185 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:47.962299 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:47.963034 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.963552 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.964447 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.964724 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.964831 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.967307 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:47.967310 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:47.967313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:47.967340 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:47.968617 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:47.969048 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:47.969052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:47.969054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:47.969138 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:47.969436 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:47.969440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:47.969442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:47.969628 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:47.969768 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:47.970016 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 00:48:47.970135 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 00:48:47.973397 18451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:48:47.976531 18451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:48:47.991394 19256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.985329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:46793 (local address 127.0.0.1:44046)
0806 00:48:47.986643 (+ 1314us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:47.986654 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:47.986771 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:47.987246 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:47.987248 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:47.987265 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:47.987376 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:47.987383 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.988533 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.988536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:47.989744 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:47.989754 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.990825 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.990828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:47.990941 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.990994 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 00:48:47.991103 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 00:48:47.991216 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1248,"thread_start_us":119,"threads_started":1}
I20260806 00:48:47.991580 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.986005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44046 (local address 127.15.147.130:46793)
0806 00:48:47.986558 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:47.986572 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:47.986746 (+ 174us) server_negotiation.cc:407] Connection header received
0806 00:48:47.986920 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:47.986927 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:47.987115 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:47.987266 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:47.987507 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.988397 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.989869 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.990690 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.990891 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.991030 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:47.991146 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 00:48:47.991245 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":102,"threads_started":1}
I20260806 00:48:47.991780 19255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.985342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57812 (local address 127.15.147.129:40491)
0806 00:48:47.985761 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:47.985776 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:47.986208 (+ 432us) server_negotiation.cc:407] Connection header received
0806 00:48:47.986521 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:47.986528 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:47.986728 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:47.986832 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:47.987292 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.988511 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.990070 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.990727 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.990841 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.990983 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:47.991325 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 00:48:47.991420 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":92,"threads_started":1}
I20260806 00:48:47.992172 19257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.985191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:40491 (local address 127.0.0.1:57812)
0806 00:48:47.986062 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:47.986113 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:47.986320 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:47.986993 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:47.986998 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:47.987016 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:47.987158 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:47.987166 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.988656 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.988659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:47.989929 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:47.989940 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.991086 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.991092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:47.991175 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.991194 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 00:48:47.991961 (+ 767us) client_negotiation.cc:240] Negotiation successful
0806 00:48:47.992035 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"mutex_wait_us":3,"thread_start_us":76,"threads_started":1}
I20260806 00:48:47.994415 18909 tablet_service.cc:1467] Processing CreateTablet for tablet bdd14b49a17346e280e2fede6c841e79 (DEFAULT_TABLE table=test-table [id=cd0cf6aa69f64d1ebfcebf4b053c8d7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:47.994508 18675 tablet_service.cc:1467] Processing CreateTablet for tablet bdd14b49a17346e280e2fede6c841e79 (DEFAULT_TABLE table=test-table [id=cd0cf6aa69f64d1ebfcebf4b053c8d7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:47.995247 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.986060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60416 (local address 127.15.147.131:44909)
0806 00:48:47.986584 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:47.986596 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:47.987119 (+ 523us) server_negotiation.cc:407] Connection header received
0806 00:48:47.987204 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:47.987212 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:47.987415 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:47.987553 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:47.987780 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.988887 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.990249 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.991036 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.991186 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.991296 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:47.994967 (+ 3671us) server_negotiation.cc:299] Negotiation successful
0806 00:48:47.995080 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":123,"threads_started":1}
I20260806 00:48:47.995478 18909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdd14b49a17346e280e2fede6c841e79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:47.995483 19254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:47.985601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:44909 (local address 127.0.0.1:60416)
0806 00:48:47.987005 (+ 1404us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:47.987017 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:47.987104 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:47.987539 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:47.987542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:47.987565 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:47.987659 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:47.987665 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.989016 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.989019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:47.990129 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:47.990138 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:47.991362 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:47.991366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:47.991475 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:47.991507 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 00:48:47.991589 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 00:48:47.991675 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1344,"thread_start_us":102,"threads_started":1}
I20260806 00:48:47.995481 18675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdd14b49a17346e280e2fede6c841e79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:47.997954 19108 tablet_service.cc:1467] Processing CreateTablet for tablet bdd14b49a17346e280e2fede6c841e79 (DEFAULT_TABLE table=test-table [id=cd0cf6aa69f64d1ebfcebf4b053c8d7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:47.998896 19108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdd14b49a17346e280e2fede6c841e79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:48.002293 19263 tablet_bootstrap.cc:492] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388: Bootstrap starting.
I20260806 00:48:48.004643 19263 tablet_bootstrap.cc:654] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:48.004630 19264 tablet_bootstrap.cc:492] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c: Bootstrap starting.
I20260806 00:48:48.005405 19263 log.cc:824] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:48.006729 19264 tablet_bootstrap.cc:654] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:48.007189 19266 tablet_bootstrap.cc:492] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4: Bootstrap starting.
I20260806 00:48:48.007543 19263 tablet_bootstrap.cc:492] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388: No bootstrap required, opened a new log
I20260806 00:48:48.007737 19263 ts_tablet_manager.cc:1397] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:48:48.008059 19264 log.cc:824] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:48.009267 19266 tablet_bootstrap.cc:654] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:48.009724 19264 tablet_bootstrap.cc:492] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c: No bootstrap required, opened a new log
I20260806 00:48:48.009917 19264 ts_tablet_manager.cc:1397] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:48:48.010085 19266 log.cc:824] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:48.011811 19263 raft_consensus.cc:348] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.012094 19263 raft_consensus.cc:374] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:48.012116 19266 tablet_bootstrap.cc:492] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4: No bootstrap required, opened a new log
I20260806 00:48:48.012161 19263 raft_consensus.cc:729] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a5b0afc8c84409981faf15563753388, State: Initialized, Role: FOLLOWER
I20260806 00:48:48.012282 19266 ts_tablet_manager.cc:1397] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:48:48.012705 19263 consensus_queue.cc:260] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.013447 19020 heartbeater.cc:503] Master 127.15.147.190:33337 was elected leader, sending a full tablet report...
I20260806 00:48:48.013396 19264 raft_consensus.cc:348] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.013633 19264 raft_consensus.cc:374] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:48.013679 19264 raft_consensus.cc:729] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13b69f2f7f684c4bae51d211aee4ac9c, State: Initialized, Role: FOLLOWER
I20260806 00:48:48.013775 19263 ts_tablet_manager.cc:1428] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:48:48.014065 19264 consensus_queue.cc:260] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.014737 18794 heartbeater.cc:503] Master 127.15.147.190:33337 was elected leader, sending a full tablet report...
I20260806 00:48:48.015020 19264 ts_tablet_manager.cc:1428] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 00:48:48.017023 19266 raft_consensus.cc:348] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.017269 19266 raft_consensus.cc:374] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:48.017333 19266 raft_consensus.cc:729] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 944d83c0514b408482a985429baa67c4, State: Initialized, Role: FOLLOWER
I20260806 00:48:48.017742 19266 consensus_queue.cc:260] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.018353 19227 heartbeater.cc:503] Master 127.15.147.190:33337 was elected leader, sending a full tablet report...
I20260806 00:48:48.018687 19266 ts_tablet_manager.cc:1428] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260806 00:48:48.027804 19271 raft_consensus.cc:482] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:48:48.028034 19271 raft_consensus.cc:504] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.029201 19271 leader_election.cc:288] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13b69f2f7f684c4bae51d211aee4ac9c (127.15.147.129:40491), 944d83c0514b408482a985429baa67c4 (127.15.147.131:44909)
I20260806 00:48:48.033459 18838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:48.029486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:40491 (local address 127.15.147.130:45747)
0806 00:48:48.029696 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:48.029712 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:48.029821 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:48.030184 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:48.030187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:48.030206 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:48.030336 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:48.030342 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.031183 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.031186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:48.032207 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:48.032222 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.032998 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.033002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:48.033118 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:48.033140 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:48:48.033242 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 00:48:48.033326 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260806 00:48:48.033459 19255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:48.029553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:45747 (local address 127.15.147.129:40491)
0806 00:48:48.029680 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:48.029684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:48.029793 (+ 109us) server_negotiation.cc:407] Connection header received
0806 00:48:48.029988 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:48.029991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:48.030068 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:48.030169 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:48.030454 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.030993 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.032383 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.032866 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.033003 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:48.033063 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:48.033259 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 00:48:48.033330 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:48:48.033954 19275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:48.029486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:44909 (local address 127.15.147.130:51163)
0806 00:48:48.029952 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:48.029968 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:48.030119 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:48.030408 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:48.030411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:48.030427 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:48.031070 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:48.031080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.031874 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.031881 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:48.032847 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:48.032857 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.033529 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.033533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:48.033637 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:48.033659 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:48:48.033732 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 00:48:48.033795 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":33,"thread_start_us":135,"threads_started":1}
I20260806 00:48:48.033989 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:48.029558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:51163 (local address 127.15.147.131:44909)
0806 00:48:48.029811 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:48.029817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:48.030043 (+ 226us) server_negotiation.cc:407] Connection header received
0806 00:48:48.030247 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:48.030250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:48.030299 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:48.030391 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:48.031195 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.031758 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.032975 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.033385 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.033519 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:48.033567 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:48.033762 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 00:48:48.033869 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260806 00:48:48.034394 18712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdd14b49a17346e280e2fede6c841e79" candidate_uuid: "4a5b0afc8c84409981faf15563753388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" is_pre_election: true
I20260806 00:48:48.034701 19161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdd14b49a17346e280e2fede6c841e79" candidate_uuid: "4a5b0afc8c84409981faf15563753388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "944d83c0514b408482a985429baa67c4" is_pre_election: true
I20260806 00:48:48.034739 18712 raft_consensus.cc:2454] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a5b0afc8c84409981faf15563753388 in term 0.
I20260806 00:48:48.034968 19161 raft_consensus.cc:2454] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a5b0afc8c84409981faf15563753388 in term 0.
I20260806 00:48:48.035396 18830 leader_election.cc:302] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13b69f2f7f684c4bae51d211aee4ac9c, 4a5b0afc8c84409981faf15563753388; no voters:
I20260806 00:48:48.035699 19271 raft_consensus.cc:2781] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:48:48.035776 19271 raft_consensus.cc:482] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:48:48.035807 19271 raft_consensus.cc:3037] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:48.037045 19271 raft_consensus.cc:504] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.037421 19271 leader_election.cc:288] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [CANDIDATE]: Term 1 election: Requested vote from peers 13b69f2f7f684c4bae51d211aee4ac9c (127.15.147.129:40491), 944d83c0514b408482a985429baa67c4 (127.15.147.131:44909)
I20260806 00:48:48.037751 18712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdd14b49a17346e280e2fede6c841e79" candidate_uuid: "4a5b0afc8c84409981faf15563753388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13b69f2f7f684c4bae51d211aee4ac9c"
I20260806 00:48:48.037781 19161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdd14b49a17346e280e2fede6c841e79" candidate_uuid: "4a5b0afc8c84409981faf15563753388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "944d83c0514b408482a985429baa67c4"
I20260806 00:48:48.037880 18712 raft_consensus.cc:3037] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:48.037895 19161 raft_consensus.cc:3037] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:48.038913 19161 raft_consensus.cc:2454] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a5b0afc8c84409981faf15563753388 in term 1.
I20260806 00:48:48.038931 18712 raft_consensus.cc:2454] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a5b0afc8c84409981faf15563753388 in term 1.
I20260806 00:48:48.039250 18827 leader_election.cc:302] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a5b0afc8c84409981faf15563753388, 944d83c0514b408482a985429baa67c4; no voters:
I20260806 00:48:48.039451 19271 raft_consensus.cc:2781] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:48.039692 19271 raft_consensus.cc:686] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [term 1 LEADER]: Becoming Leader. State: Replica: 4a5b0afc8c84409981faf15563753388, State: Running, Role: LEADER
I20260806 00:48:48.040006 19271 consensus_queue.cc:237] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.043053 18451 catalog_manager.cc:5495] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a5b0afc8c84409981faf15563753388 (127.15.147.130). New cstate: current_term: 1 leader_uuid: "4a5b0afc8c84409981faf15563753388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } health_report { overall_health: UNKNOWN } } }
Aug 06 00:48:48 dist-test-slave-45dx krb5kdc[18385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977327, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:48.103145 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:48.094274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38402 (local address 127.15.147.190:33337)
0806 00:48:48.094442 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:48.094447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:48.094724 (+ 277us) server_negotiation.cc:407] Connection header received
0806 00:48:48.094846 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:48.094848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:48.094886 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:48.094948 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:48.095622 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.096113 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.096966 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.097149 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.097219 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:48.099482 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:48.099485 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:48.099487 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:48.099507 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:48.101017 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:48.101597 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:48.101600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:48.101601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:48.101676 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:48.102134 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:48.102136 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:48.102137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:48.102291 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:48.102463 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:48.102767 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 00:48:48.102927 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":79}
I20260806 00:48:48.103145 19284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:48.094159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:33337 (local address 127.0.0.1:38402)
0806 00:48:48.094467 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:48.094480 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:48.094764 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:48.094995 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:48.094997 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:48.095424 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:48.095523 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:48.095529 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.096247 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.096250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:48.096869 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:48.096877 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.097243 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.097246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:48.097317 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:48.097853 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:48.097864 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:48.099349 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:48.101119 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:48.101128 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:48.101133 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:48.101506 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:48.101773 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:48.101775 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:48.101777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:48.102038 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:48.102479 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:48.102486 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:48.102596 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 00:48:48.102718 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 00:48:48.102872 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":90,"threads_started":1}
I20260806 00:48:48.107326 18451 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:38402:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 00:48:48.108047 18451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:48:48.113826 18909 tablet_service.cc:1467] Processing CreateTablet for tablet 2f092bca163c4329a34d5257eeed93fe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=48f50b092e254a3cade99f9286c08cf9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:48:48.113859 18675 tablet_service.cc:1467] Processing CreateTablet for tablet 2f092bca163c4329a34d5257eeed93fe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=48f50b092e254a3cade99f9286c08cf9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:48:48.114639 18675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f092bca163c4329a34d5257eeed93fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:48.114638 18909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f092bca163c4329a34d5257eeed93fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:48.114431 19108 tablet_service.cc:1467] Processing CreateTablet for tablet 2f092bca163c4329a34d5257eeed93fe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=48f50b092e254a3cade99f9286c08cf9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:48:48.115123 19108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f092bca163c4329a34d5257eeed93fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:48.117949 19263 tablet_bootstrap.cc:492] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388: Bootstrap starting.
I20260806 00:48:48.119068 19263 tablet_bootstrap.cc:654] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:48.119784 19266 tablet_bootstrap.cc:492] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4: Bootstrap starting.
I20260806 00:48:48.120425 19263 tablet_bootstrap.cc:492] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388: No bootstrap required, opened a new log
I20260806 00:48:48.120493 19263 ts_tablet_manager.cc:1397] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:48:48.120608 19266 tablet_bootstrap.cc:654] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:48.120857 19263 raft_consensus.cc:348] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.120952 19263 raft_consensus.cc:374] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:48.120973 19263 raft_consensus.cc:729] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a5b0afc8c84409981faf15563753388, State: Initialized, Role: FOLLOWER
I20260806 00:48:48.121100 19263 consensus_queue.cc:260] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.121395 19263 ts_tablet_manager.cc:1428] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 00:48:48.121368 19271 tablet_replica.cc:439] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } } }
I20260806 00:48:48.121521 19271 tablet_replica.cc:442] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:48:48.121814 19266 tablet_bootstrap.cc:492] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4: No bootstrap required, opened a new log
I20260806 00:48:48.121929 19266 ts_tablet_manager.cc:1397] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 00:48:48.122188 19264 tablet_bootstrap.cc:492] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c: Bootstrap starting.
I20260806 00:48:48.122447 19266 raft_consensus.cc:348] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.122622 19266 raft_consensus.cc:374] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:48.122658 19266 raft_consensus.cc:729] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 944d83c0514b408482a985429baa67c4, State: Initialized, Role: FOLLOWER
I20260806 00:48:48.122784 19266 consensus_queue.cc:260] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.123113 19266 ts_tablet_manager.cc:1428] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 00:48:48.123534 19264 tablet_bootstrap.cc:654] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:48.123507 19274 tablet_replica.cc:439] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } } }
I20260806 00:48:48.123876 19274 tablet_replica.cc:442] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:48:48.125088 19264 tablet_bootstrap.cc:492] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c: No bootstrap required, opened a new log
I20260806 00:48:48.125203 19264 ts_tablet_manager.cc:1397] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:48:48.125790 19264 raft_consensus.cc:348] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.125947 19264 raft_consensus.cc:374] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:48.126012 19264 raft_consensus.cc:729] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13b69f2f7f684c4bae51d211aee4ac9c, State: Initialized, Role: FOLLOWER
I20260806 00:48:48.126179 19264 consensus_queue.cc:260] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.126483 19264 ts_tablet_manager.cc:1428] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 00:48:48.126505 19272 tablet_replica.cc:439] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } } }
I20260806 00:48:48.126642 19272 tablet_replica.cc:442] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:48:48.135471 19274 raft_consensus.cc:482] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:48:48.135556 19274 raft_consensus.cc:504] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.136538 19274 leader_election.cc:288] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13b69f2f7f684c4bae51d211aee4ac9c (127.15.147.129:40491), 4a5b0afc8c84409981faf15563753388 (127.15.147.130:46793)
I20260806 00:48:48.140057 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:48.136818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:43281 (local address 127.15.147.130:46793)
0806 00:48:48.137134 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:48.137139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:48.137151 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:48:48.137195 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:48.137200 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:48.137248 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:48.137334 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:48.137613 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.138096 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.139085 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.139485 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.139610 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:48.139660 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:48.139863 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 00:48:48.139937 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235}
I20260806 00:48:48.140080 19074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:48.136734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:46793 (local address 127.15.147.131:43281)
0806 00:48:48.136890 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:48.136904 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:48.136978 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:48.137355 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:48.137358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:48.137375 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:48.137487 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:48.137492 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.138212 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.138216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:48.138961 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:48.138969 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.139609 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.139613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:48.139733 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:48.139751 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 00:48:48.139835 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 00:48:48.139936 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 00:48:48.140560 19255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:48.137055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:37397 (local address 127.15.147.129:40491)
0806 00:48:48.137167 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:48.137171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:48.137293 (+ 122us) server_negotiation.cc:407] Connection header received
0806 00:48:48.137420 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:48.137423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:48.137467 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:48.137560 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:48.137849 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.138369 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.139502 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.139916 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.140049 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:48.140101 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:48.140374 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 00:48:48.140444 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:48:48.140591 19290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:48.136748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:40491 (local address 127.15.147.131:37397)
0806 00:48:48.137186 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:48.137200 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:48.137312 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:48.137585 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:48.137588 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:48.137605 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:48.137736 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:48.137741 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.138486 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.138490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:48.139368 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:48.139378 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.140067 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.140072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:48.140190 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:48.140220 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 00:48:48.140361 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 00:48:48.140436 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":4,"thread_start_us":177,"threads_started":1}
I20260806 00:48:48.140758 18942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f092bca163c4329a34d5257eeed93fe" candidate_uuid: "944d83c0514b408482a985429baa67c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a5b0afc8c84409981faf15563753388" is_pre_election: true
I20260806 00:48:48.140973 18942 raft_consensus.cc:2454] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 944d83c0514b408482a985429baa67c4 in term 0.
I20260806 00:48:48.140954 18712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f092bca163c4329a34d5257eeed93fe" candidate_uuid: "944d83c0514b408482a985429baa67c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" is_pre_election: true
I20260806 00:48:48.141070 18712 raft_consensus.cc:2454] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 944d83c0514b408482a985429baa67c4 in term 0.
I20260806 00:48:48.141467 19068 leader_election.cc:302] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 13b69f2f7f684c4bae51d211aee4ac9c, 4a5b0afc8c84409981faf15563753388, 944d83c0514b408482a985429baa67c4; no voters:
I20260806 00:48:48.141767 19274 raft_consensus.cc:2781] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:48:48.141832 19274 raft_consensus.cc:482] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:48:48.141872 19274 raft_consensus.cc:3037] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:48.142793 19274 raft_consensus.cc:504] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.143153 19274 leader_election.cc:288] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [CANDIDATE]: Term 1 election: Requested vote from peers 13b69f2f7f684c4bae51d211aee4ac9c (127.15.147.129:40491), 4a5b0afc8c84409981faf15563753388 (127.15.147.130:46793)
I20260806 00:48:48.143513 18712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f092bca163c4329a34d5257eeed93fe" candidate_uuid: "944d83c0514b408482a985429baa67c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13b69f2f7f684c4bae51d211aee4ac9c"
I20260806 00:48:48.143572 18942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f092bca163c4329a34d5257eeed93fe" candidate_uuid: "944d83c0514b408482a985429baa67c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a5b0afc8c84409981faf15563753388"
I20260806 00:48:48.143646 18712 raft_consensus.cc:3037] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:48.143676 18942 raft_consensus.cc:3037] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:48.145053 18712 raft_consensus.cc:2454] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 944d83c0514b408482a985429baa67c4 in term 1.
I20260806 00:48:48.145053 18942 raft_consensus.cc:2454] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 944d83c0514b408482a985429baa67c4 in term 1.
I20260806 00:48:48.145427 19065 leader_election.cc:302] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a5b0afc8c84409981faf15563753388, 944d83c0514b408482a985429baa67c4; no voters:
I20260806 00:48:48.145630 19274 raft_consensus.cc:2781] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:48.145846 19274 raft_consensus.cc:686] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [term 1 LEADER]: Becoming Leader. State: Replica: 944d83c0514b408482a985429baa67c4, State: Running, Role: LEADER
I20260806 00:48:48.146173 19274 consensus_queue.cc:237] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.148381 19291 tablet_replica.cc:439] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4: TxnStatusTablet state changed. Reason: New leader 944d83c0514b408482a985429baa67c4. Latest consensus state: current_term: 1 leader_uuid: "944d83c0514b408482a985429baa67c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } } }
I20260806 00:48:48.148491 19291 tablet_replica.cc:442] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:48:48.148782 19294 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 00:48:48.148989 19294 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 00:48:48.149273 18451 catalog_manager.cc:5495] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 944d83c0514b408482a985429baa67c4 (127.15.147.131). New cstate: current_term: 1 leader_uuid: "944d83c0514b408482a985429baa67c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } health_report { overall_health: HEALTHY } } }
I20260806 00:48:48.162528 19284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:48.158360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:44909 (local address 127.0.0.1:60418)
0806 00:48:48.158636 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:48.158649 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:48.158738 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:48.159120 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:48.159123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:48.159130 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:48:48.159219 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:48.159224 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.160249 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.160251 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:48.160942 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:48.160951 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.161447 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.161450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:48.161513 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:48.161555 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:48:48.162290 (+ 735us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:48:48.162302 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 00:48:48.162352 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 00:48:48.162423 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188}
I20260806 00:48:48.162561 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:48.158486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60418 (local address 127.15.147.131:44909)
0806 00:48:48.158619 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:48.158624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:48.158691 (+ 67us) server_negotiation.cc:407] Connection header received
0806 00:48:48.158880 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:48.158884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:48.158950 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:48.159081 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:48.159330 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.160138 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.161035 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.161346 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.161449 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:48.161686 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:48.162192 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:48.162311 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:48.162371 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 00:48:48.162426 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:48:48.167912 18942 raft_consensus.cc:1261] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388 [term 1 FOLLOWER]: Refusing update from remote peer 944d83c0514b408482a985429baa67c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:48.168012 18712 raft_consensus.cc:1261] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c [term 1 FOLLOWER]: Refusing update from remote peer 944d83c0514b408482a985429baa67c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:48.168561 19274 consensus_queue.cc:1035] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:48.168731 19291 consensus_queue.cc:1035] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:48.172384 19272 tablet_replica.cc:439] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c: TxnStatusTablet state changed. Reason: New leader 944d83c0514b408482a985429baa67c4. Latest consensus state: current_term: 1 leader_uuid: "944d83c0514b408482a985429baa67c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } } }
I20260806 00:48:48.172539 19272 tablet_replica.cc:442] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:48:48.173606 19271 tablet_replica.cc:439] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388: TxnStatusTablet state changed. Reason: New leader 944d83c0514b408482a985429baa67c4. Latest consensus state: current_term: 1 leader_uuid: "944d83c0514b408482a985429baa67c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } } }
I20260806 00:48:48.173725 19271 tablet_replica.cc:442] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 00:48:48.176600 19228 tablet.cc:2369] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:48:48.176818 19274 tablet_replica.cc:439] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "944d83c0514b408482a985429baa67c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } } }
I20260806 00:48:48.176934 19274 tablet_replica.cc:442] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:48:48.178229 19271 tablet_replica.cc:439] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "944d83c0514b408482a985429baa67c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } } }
I20260806 00:48:48.178351 19271 tablet_replica.cc:442] T 2f092bca163c4329a34d5257eeed93fe P 4a5b0afc8c84409981faf15563753388: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:48:48.178619 19274 tablet_replica.cc:439] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "944d83c0514b408482a985429baa67c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } } }
I20260806 00:48:48.178714 19274 tablet_replica.cc:442] T 2f092bca163c4329a34d5257eeed93fe P 944d83c0514b408482a985429baa67c4: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:48:48.180028 19299 mvcc.cc:205] Tried to move back new op lower bound from 7315363136169074688 to 7315363136089133056. Current Snapshot: MvccSnapshot[applied={T|T < 7315363136169074688}]
I20260806 00:48:48.180076 19272 tablet_replica.cc:439] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "944d83c0514b408482a985429baa67c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } } }
I20260806 00:48:48.180188 19272 tablet_replica.cc:442] T 2f092bca163c4329a34d5257eeed93fe P 13b69f2f7f684c4bae51d211aee4ac9c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:48:48.181739 19300 mvcc.cc:205] Tried to move back new op lower bound from 7315363136169074688 to 7315363136089133056. Current Snapshot: MvccSnapshot[applied={T|T < 7315363136169074688}]
I20260806 00:48:48.188143 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:48.184348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44056 (local address 127.15.147.130:46793)
0806 00:48:48.184472 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:48.184476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:48.184490 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:48:48.184672 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:48.184674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:48.184719 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:48.184795 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:48.185121 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.185601 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.186826 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.187033 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.187106 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:48.187432 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:48.187781 (+ 349us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:48.187859 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:48.187955 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 00:48:48.187995 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:48:48.198570 19034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:48.195280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:44909 (local address 127.15.147.130:46967)
0806 00:48:48.195435 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:48.195456 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:48.195521 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:48.195733 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:48.195735 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:48.195740 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:48:48.195832 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:48.195836 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.196818 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.196820 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:48.197369 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:48.197377 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.197919 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.197921 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:48.197989 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:48.198027 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:48:48.198351 (+ 324us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:48:48.198355 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 00:48:48.198414 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 00:48:48.198472 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 00:48:48.198601 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:48.195380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:46967 (local address 127.15.147.131:44909)
0806 00:48:48.195498 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:48.195502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:48.195516 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:48:48.195591 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:48.195594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:48.195641 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:48.195719 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:48.195931 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.196689 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.197481 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.197805 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.197914 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:48.198164 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:48.198248 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:48.198338 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:48.198428 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 00:48:48.198476 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:48:48.206648 18712 raft_consensus.cc:1261] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c [term 1 FOLLOWER]: Refusing update from remote peer 4a5b0afc8c84409981faf15563753388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:48.206741 19161 raft_consensus.cc:1261] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4 [term 1 FOLLOWER]: Refusing update from remote peer 4a5b0afc8c84409981faf15563753388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:48.207150 19276 consensus_queue.cc:1035] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:48.207371 19271 consensus_queue.cc:1035] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 00:48:48.209113 19022 tablet.cc:2369] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:48:48.233667 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:48.229999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:48357 (local address 127.15.147.130:46793)
0806 00:48:48.230098 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:48.230102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:48.230143 (+ 41us) server_negotiation.cc:407] Connection header received
0806 00:48:48.230271 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:48.230274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:48.230322 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:48.230401 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:48.230684 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.231437 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.232646 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.232850 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.232933 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:48.233279 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:48.233344 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:48.233400 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:48.233523 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 00:48:48.233570 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 00:48:48.233752 19237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:48.229912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:46793 (local address 127.15.147.131:48357)
0806 00:48:48.230076 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:48.230092 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:48.230178 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:48.230416 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:48.230418 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:48.230423 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:48:48.230564 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:48.230570 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.231574 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.231578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:48.232529 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:48.232541 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:48.232972 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:48.232995 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:48.233093 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:48.233147 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:48:48.233428 (+ 281us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:48:48.233435 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 00:48:48.233505 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 00:48:48.233584 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 00:48:48.260550 18451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:48:48.260742 18451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:48:48.263530 18451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0: Sending DeleteTablet for 3 replicas of tablet bdd14b49a17346e280e2fede6c841e79
I20260806 00:48:48.264868 18675 tablet_service.cc:1514] Processing DeleteTablet for tablet bdd14b49a17346e280e2fede6c841e79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:48:48 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:57812
I20260806 00:48:48.265251 18909 tablet_service.cc:1514] Processing DeleteTablet for tablet bdd14b49a17346e280e2fede6c841e79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:48:48 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:44046
I20260806 00:48:48.265843 18451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:48:48.266263 18451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:48:48.266321 19324 tablet_replica.cc:330] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c: stopping tablet replica
I20260806 00:48:48.268350 19324 raft_consensus.cc:2229] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:48:48.268582 19324 raft_consensus.cc:2258] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:48:48.269073 19108 tablet_service.cc:1514] Processing DeleteTablet for tablet bdd14b49a17346e280e2fede6c841e79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:48:48 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:60416
I20260806 00:48:48.270721 19327 tablet_replica.cc:330] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4: stopping tablet replica
I20260806 00:48:48.271075 19327 raft_consensus.cc:2229] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:48:48.271293 19327 raft_consensus.cc:2258] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:48:48.271292 19324 ts_tablet_manager.cc:1905] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:48:48.272648 19327 ts_tablet_manager.cc:1905] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:48:48.274775 18675 tablet_service.cc:1467] Processing CreateTablet for tablet 1b7c3e4db29144cc9e99771eafa5d682 (DEFAULT_TABLE table=test-table [id=bebff4bac81a4764b842f05a70767bc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:48.275071 18675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b7c3e4db29144cc9e99771eafa5d682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:48.275426 18909 tablet_service.cc:1467] Processing CreateTablet for tablet 1b7c3e4db29144cc9e99771eafa5d682 (DEFAULT_TABLE table=test-table [id=bebff4bac81a4764b842f05a70767bc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:48.275723 18909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b7c3e4db29144cc9e99771eafa5d682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:48.276257 19327 ts_tablet_manager.cc:1918] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:48:48.276433 19327 log.cc:1196] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/bdd14b49a17346e280e2fede6c841e79
I20260806 00:48:48.276881 19327 ts_tablet_manager.cc:1939] T bdd14b49a17346e280e2fede6c841e79 P 944d83c0514b408482a985429baa67c4: Deleting consensus metadata
I20260806 00:48:48.277385 19264 tablet_bootstrap.cc:492] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c: Bootstrap starting.
I20260806 00:48:48.278357 19264 tablet_bootstrap.cc:654] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:48.278352 19108 tablet_service.cc:1467] Processing CreateTablet for tablet 1b7c3e4db29144cc9e99771eafa5d682 (DEFAULT_TABLE table=test-table [id=bebff4bac81a4764b842f05a70767bc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:48.278633 19108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b7c3e4db29144cc9e99771eafa5d682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:48.275065 19324 ts_tablet_manager.cc:1918] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:48:48.279313 19324 log.cc:1196] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/wal/wals/bdd14b49a17346e280e2fede6c841e79
I20260806 00:48:48.279663 19324 ts_tablet_manager.cc:1939] T bdd14b49a17346e280e2fede6c841e79 P 13b69f2f7f684c4bae51d211aee4ac9c: Deleting consensus metadata
I20260806 00:48:48.280495 19263 tablet_bootstrap.cc:492] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388: Bootstrap starting.
I20260806 00:48:48.280866 18429 catalog_manager.cc:4841] TS 13b69f2f7f684c4bae51d211aee4ac9c (127.15.147.129:40491): tablet bdd14b49a17346e280e2fede6c841e79 (table test-table [id=cd0cf6aa69f64d1ebfcebf4b053c8d7e]) successfully deleted
I20260806 00:48:48.281073 19266 tablet_bootstrap.cc:492] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4: Bootstrap starting.
I20260806 00:48:48.281687 19263 tablet_bootstrap.cc:654] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:48.281807 19264 tablet_bootstrap.cc:492] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c: No bootstrap required, opened a new log
I20260806 00:48:48.281888 19264 ts_tablet_manager.cc:1397] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 00:48:48.282148 19266 tablet_bootstrap.cc:654] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:48.282418 19264 raft_consensus.cc:348] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.282548 19264 raft_consensus.cc:374] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:48.282582 19264 raft_consensus.cc:729] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13b69f2f7f684c4bae51d211aee4ac9c, State: Initialized, Role: FOLLOWER
I20260806 00:48:48.282734 19264 consensus_queue.cc:260] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.282809 18424 catalog_manager.cc:4841] TS 944d83c0514b408482a985429baa67c4 (127.15.147.131:44909): tablet bdd14b49a17346e280e2fede6c841e79 (table test-table [id=cd0cf6aa69f64d1ebfcebf4b053c8d7e]) successfully deleted
I20260806 00:48:48.283015 19264 ts_tablet_manager.cc:1428] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 00:48:48.283881 19263 tablet_bootstrap.cc:492] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388: No bootstrap required, opened a new log
I20260806 00:48:48.283991 19263 ts_tablet_manager.cc:1397] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 00:48:48.284524 19263 raft_consensus.cc:348] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.284649 19263 raft_consensus.cc:374] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:48.284695 19263 raft_consensus.cc:729] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a5b0afc8c84409981faf15563753388, State: Initialized, Role: FOLLOWER
I20260806 00:48:48.284859 19263 consensus_queue.cc:260] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.285181 19263 ts_tablet_manager.cc:1428] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:48:48.285970 19266 tablet_bootstrap.cc:492] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4: No bootstrap required, opened a new log
I20260806 00:48:48.286075 19266 ts_tablet_manager.cc:1397] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260806 00:48:48.286847 19266 raft_consensus.cc:348] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.286979 19266 raft_consensus.cc:374] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:48.287024 19266 raft_consensus.cc:729] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 944d83c0514b408482a985429baa67c4, State: Initialized, Role: FOLLOWER
I20260806 00:48:48.287148 19266 consensus_queue.cc:260] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.287433 19266 ts_tablet_manager.cc:1428] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:48:48.289103 19326 tablet_replica.cc:330] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388: stopping tablet replica
I20260806 00:48:48.289440 19326 raft_consensus.cc:2229] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:48:48.289765 19326 raft_consensus.cc:2258] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:48:48.291345 19326 ts_tablet_manager.cc:1905] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:48:48.292084 19316 raft_consensus.cc:482] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:48:48.292161 19316 raft_consensus.cc:504] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.292550 19316 leader_election.cc:288] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13b69f2f7f684c4bae51d211aee4ac9c (127.15.147.129:40491), 944d83c0514b408482a985429baa67c4 (127.15.147.131:44909)
I20260806 00:48:48.293068 18711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b7c3e4db29144cc9e99771eafa5d682" candidate_uuid: "4a5b0afc8c84409981faf15563753388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" is_pre_election: true
I20260806 00:48:48.293076 19161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b7c3e4db29144cc9e99771eafa5d682" candidate_uuid: "4a5b0afc8c84409981faf15563753388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "944d83c0514b408482a985429baa67c4" is_pre_election: true
I20260806 00:48:48.293184 18711 raft_consensus.cc:2454] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a5b0afc8c84409981faf15563753388 in term 0.
I20260806 00:48:48.293184 19161 raft_consensus.cc:2454] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a5b0afc8c84409981faf15563753388 in term 0.
I20260806 00:48:48.293478 18827 leader_election.cc:302] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a5b0afc8c84409981faf15563753388, 944d83c0514b408482a985429baa67c4; no voters:
I20260806 00:48:48.293674 19316 raft_consensus.cc:2781] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:48:48.293726 19316 raft_consensus.cc:482] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:48:48.293762 19316 raft_consensus.cc:3037] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:48.295290 19316 raft_consensus.cc:504] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.295646 19316 leader_election.cc:288] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [CANDIDATE]: Term 1 election: Requested vote from peers 13b69f2f7f684c4bae51d211aee4ac9c (127.15.147.129:40491), 944d83c0514b408482a985429baa67c4 (127.15.147.131:44909)
I20260806 00:48:48.295892 18711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b7c3e4db29144cc9e99771eafa5d682" candidate_uuid: "4a5b0afc8c84409981faf15563753388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13b69f2f7f684c4bae51d211aee4ac9c"
I20260806 00:48:48.296025 18711 raft_consensus.cc:3037] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:48.296042 19161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b7c3e4db29144cc9e99771eafa5d682" candidate_uuid: "4a5b0afc8c84409981faf15563753388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "944d83c0514b408482a985429baa67c4"
I20260806 00:48:48.296139 19161 raft_consensus.cc:3037] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:48.297257 18711 raft_consensus.cc:2454] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a5b0afc8c84409981faf15563753388 in term 1.
I20260806 00:48:48.297257 19161 raft_consensus.cc:2454] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a5b0afc8c84409981faf15563753388 in term 1.
I20260806 00:48:48.297545 18830 leader_election.cc:302] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13b69f2f7f684c4bae51d211aee4ac9c, 4a5b0afc8c84409981faf15563753388; no voters:
I20260806 00:48:48.297694 19316 raft_consensus.cc:2781] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:48.297777 19316 raft_consensus.cc:686] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [term 1 LEADER]: Becoming Leader. State: Replica: 4a5b0afc8c84409981faf15563753388, State: Running, Role: LEADER
I20260806 00:48:48.297881 19316 consensus_queue.cc:237] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:48.299547 18450 catalog_manager.cc:5495] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a5b0afc8c84409981faf15563753388 (127.15.147.130). New cstate: current_term: 1 leader_uuid: "4a5b0afc8c84409981faf15563753388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } health_report { overall_health: UNKNOWN } } }
I20260806 00:48:48.301707 19326 ts_tablet_manager.cc:1918] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:48:48.301872 19326 log.cc:1196] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/bdd14b49a17346e280e2fede6c841e79
I20260806 00:48:48.302237 19326 ts_tablet_manager.cc:1939] T bdd14b49a17346e280e2fede6c841e79 P 4a5b0afc8c84409981faf15563753388: Deleting consensus metadata
I20260806 00:48:48.303520 18426 catalog_manager.cc:4841] TS 4a5b0afc8c84409981faf15563753388 (127.15.147.130:46793): tablet bdd14b49a17346e280e2fede6c841e79 (table test-table [id=cd0cf6aa69f64d1ebfcebf4b053c8d7e]) successfully deleted
I20260806 00:48:48.321022 19161 raft_consensus.cc:1261] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4 [term 1 FOLLOWER]: Refusing update from remote peer 4a5b0afc8c84409981faf15563753388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:48.321022 18711 raft_consensus.cc:1261] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c [term 1 FOLLOWER]: Refusing update from remote peer 4a5b0afc8c84409981faf15563753388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:48.321499 19316 consensus_queue.cc:1035] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:48.321866 19316 consensus_queue.cc:1035] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:48.331221 19298 mvcc.cc:205] Tried to move back new op lower bound from 7315363136798367744 to 7315363136709689344. Current Snapshot: MvccSnapshot[applied={T|T < 7315363136798367744}]
W20260806 00:48:48.344936 19299 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 00:48:49.904435 18451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:48:49.904585 18451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:48:49.907171 18451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0: Sending DeleteTablet for 3 replicas of tablet 1b7c3e4db29144cc9e99771eafa5d682
I20260806 00:48:49.907761 18675 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b7c3e4db29144cc9e99771eafa5d682 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:48:49 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:57812
I20260806 00:48:49.907840 18909 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b7c3e4db29144cc9e99771eafa5d682 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:48:49 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:44046
I20260806 00:48:49.907860 19108 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b7c3e4db29144cc9e99771eafa5d682 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:48:49 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:60416
I20260806 00:48:49.908129 19385 tablet_replica.cc:330] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388: stopping tablet replica
I20260806 00:48:49.908277 19386 tablet_replica.cc:330] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4: stopping tablet replica
I20260806 00:48:49.908281 19385 raft_consensus.cc:2229] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:48:49.908437 19386 raft_consensus.cc:2229] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:48:49.908535 19385 raft_consensus.cc:2258] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:48:49.908547 19386 raft_consensus.cc:2258] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:48:49.908977 19384 tablet_replica.cc:330] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c: stopping tablet replica
I20260806 00:48:49.909102 19384 raft_consensus.cc:2229] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:48:49.909209 19384 raft_consensus.cc:2258] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:48:49.909220 19386 ts_tablet_manager.cc:1905] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:48:49.909842 19384 ts_tablet_manager.cc:1905] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:48:49.912361 19386 ts_tablet_manager.cc:1918] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 00:48:49.912563 19386 log.cc:1196] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/1b7c3e4db29144cc9e99771eafa5d682
I20260806 00:48:49.912654 19384 ts_tablet_manager.cc:1918] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 00:48:49.912750 19384 log.cc:1196] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/wal/wals/1b7c3e4db29144cc9e99771eafa5d682
I20260806 00:48:49.912958 19386 ts_tablet_manager.cc:1939] T 1b7c3e4db29144cc9e99771eafa5d682 P 944d83c0514b408482a985429baa67c4: Deleting consensus metadata
I20260806 00:48:49.913074 19384 ts_tablet_manager.cc:1939] T 1b7c3e4db29144cc9e99771eafa5d682 P 13b69f2f7f684c4bae51d211aee4ac9c: Deleting consensus metadata
I20260806 00:48:49.913841 18424 catalog_manager.cc:4841] TS 944d83c0514b408482a985429baa67c4 (127.15.147.131:44909): tablet 1b7c3e4db29144cc9e99771eafa5d682 (table test-table [id=bebff4bac81a4764b842f05a70767bc0]) successfully deleted
I20260806 00:48:49.913880 18429 catalog_manager.cc:4841] TS 13b69f2f7f684c4bae51d211aee4ac9c (127.15.147.129:40491): tablet 1b7c3e4db29144cc9e99771eafa5d682 (table test-table [id=bebff4bac81a4764b842f05a70767bc0]) successfully deleted
I20260806 00:48:49.914502 19385 ts_tablet_manager.cc:1905] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:48:49.918205 19385 ts_tablet_manager.cc:1918] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 00:48:49.918304 19385 log.cc:1196] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/1b7c3e4db29144cc9e99771eafa5d682
I20260806 00:48:49.918639 19385 ts_tablet_manager.cc:1939] T 1b7c3e4db29144cc9e99771eafa5d682 P 4a5b0afc8c84409981faf15563753388: Deleting consensus metadata
I20260806 00:48:49.919435 18426 catalog_manager.cc:4841] TS 4a5b0afc8c84409981faf15563753388 (127.15.147.130:46793): tablet 1b7c3e4db29144cc9e99771eafa5d682 (table test-table [id=bebff4bac81a4764b842f05a70767bc0]) successfully deleted
Aug 06 00:48:49 dist-test-slave-45dx krb5kdc[18385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977329, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 00:48:49 dist-test-slave-45dx krb5kdc[18385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977329, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:49.955472 19400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:49.946461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38416 (local address 127.15.147.190:33337)
0806 00:48:49.946745 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:49.946753 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:49.946969 (+ 216us) server_negotiation.cc:407] Connection header received
0806 00:48:49.947147 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:49.947151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:49.947205 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:49.947315 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:49.948085 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:49.948706 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:49.949545 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:49.949790 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:49.949903 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:49.952463 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:49.952466 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:49.952468 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:49.952495 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:49.953877 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:49.954330 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:49.954333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:49.954335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:49.954414 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:49.954669 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:49.954671 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:49.954672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:49.954857 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:49.954963 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:49.955210 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 00:48:49.955335 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":97,"threads_started":1}
I20260806 00:48:49.958143 18451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:48:49.958617 18451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:48:49.963379 18675 tablet_service.cc:1467] Processing CreateTablet for tablet 98dd9fa222a048eaa6ded1a9ad818625 (DEFAULT_TABLE table=test-table [id=f0f7ba2628f048d1942f925ddb85b048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:49.963503 18909 tablet_service.cc:1467] Processing CreateTablet for tablet 98dd9fa222a048eaa6ded1a9ad818625 (DEFAULT_TABLE table=test-table [id=f0f7ba2628f048d1942f925ddb85b048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:49.963551 19108 tablet_service.cc:1467] Processing CreateTablet for tablet 98dd9fa222a048eaa6ded1a9ad818625 (DEFAULT_TABLE table=test-table [id=f0f7ba2628f048d1942f925ddb85b048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:49.963757 18675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98dd9fa222a048eaa6ded1a9ad818625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:49.963816 18909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98dd9fa222a048eaa6ded1a9ad818625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:49.963838 19108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98dd9fa222a048eaa6ded1a9ad818625. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:49.966367 19401 tablet_bootstrap.cc:492] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c: Bootstrap starting.
I20260806 00:48:49.966842 19404 tablet_bootstrap.cc:492] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4: Bootstrap starting.
I20260806 00:48:49.967026 19405 tablet_bootstrap.cc:492] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388: Bootstrap starting.
I20260806 00:48:49.967572 19401 tablet_bootstrap.cc:654] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:49.967705 19404 tablet_bootstrap.cc:654] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:49.967855 19405 tablet_bootstrap.cc:654] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:49.969079 19404 tablet_bootstrap.cc:492] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4: No bootstrap required, opened a new log
I20260806 00:48:49.969151 19401 tablet_bootstrap.cc:492] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c: No bootstrap required, opened a new log
I20260806 00:48:49.969169 19404 ts_tablet_manager.cc:1397] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 00:48:49.969245 19401 ts_tablet_manager.cc:1397] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:48:49.969606 19404 raft_consensus.cc:348] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:49.969702 19404 raft_consensus.cc:374] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:49.969723 19404 raft_consensus.cc:729] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 944d83c0514b408482a985429baa67c4, State: Initialized, Role: FOLLOWER
I20260806 00:48:49.969676 19401 raft_consensus.cc:348] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:49.969820 19401 raft_consensus.cc:374] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:49.969859 19401 raft_consensus.cc:729] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13b69f2f7f684c4bae51d211aee4ac9c, State: Initialized, Role: FOLLOWER
I20260806 00:48:49.969857 19404 consensus_queue.cc:260] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:49.969996 19401 consensus_queue.cc:260] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:49.970155 19404 ts_tablet_manager.cc:1428] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 00:48:49.970427 19401 ts_tablet_manager.cc:1428] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:48:49.970722 19405 tablet_bootstrap.cc:492] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388: No bootstrap required, opened a new log
I20260806 00:48:49.970809 19405 ts_tablet_manager.cc:1397] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260806 00:48:49.971186 19405 raft_consensus.cc:348] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:49.971318 19405 raft_consensus.cc:374] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:49.971361 19405 raft_consensus.cc:729] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a5b0afc8c84409981faf15563753388, State: Initialized, Role: FOLLOWER
I20260806 00:48:49.971488 19405 consensus_queue.cc:260] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:49.971768 19405 ts_tablet_manager.cc:1428] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260806 00:48:49.989883 18795 tablet.cc:2369] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:48:50.133208 19368 raft_consensus.cc:482] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:48:50.133410 19368 raft_consensus.cc:504] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:50.133917 19368 leader_election.cc:288] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13b69f2f7f684c4bae51d211aee4ac9c (127.15.147.129:40491), 944d83c0514b408482a985429baa67c4 (127.15.147.131:44909)
I20260806 00:48:50.134397 18712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98dd9fa222a048eaa6ded1a9ad818625" candidate_uuid: "4a5b0afc8c84409981faf15563753388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" is_pre_election: true
I20260806 00:48:50.134478 19161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98dd9fa222a048eaa6ded1a9ad818625" candidate_uuid: "4a5b0afc8c84409981faf15563753388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "944d83c0514b408482a985429baa67c4" is_pre_election: true
I20260806 00:48:50.134572 18712 raft_consensus.cc:2454] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a5b0afc8c84409981faf15563753388 in term 0.
I20260806 00:48:50.134685 19161 raft_consensus.cc:2454] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a5b0afc8c84409981faf15563753388 in term 0.
I20260806 00:48:50.134989 18830 leader_election.cc:302] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13b69f2f7f684c4bae51d211aee4ac9c, 4a5b0afc8c84409981faf15563753388; no voters:
I20260806 00:48:50.135203 19368 raft_consensus.cc:2781] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:48:50.135260 19368 raft_consensus.cc:482] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:48:50.135288 19368 raft_consensus.cc:3037] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:50.136276 19368 raft_consensus.cc:504] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:50.136612 19368 leader_election.cc:288] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [CANDIDATE]: Term 1 election: Requested vote from peers 13b69f2f7f684c4bae51d211aee4ac9c (127.15.147.129:40491), 944d83c0514b408482a985429baa67c4 (127.15.147.131:44909)
I20260806 00:48:50.136994 18712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98dd9fa222a048eaa6ded1a9ad818625" candidate_uuid: "4a5b0afc8c84409981faf15563753388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13b69f2f7f684c4bae51d211aee4ac9c"
I20260806 00:48:50.137023 19161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98dd9fa222a048eaa6ded1a9ad818625" candidate_uuid: "4a5b0afc8c84409981faf15563753388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "944d83c0514b408482a985429baa67c4"
I20260806 00:48:50.137130 18712 raft_consensus.cc:3037] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:50.137137 19161 raft_consensus.cc:3037] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:50.138033 19161 raft_consensus.cc:2454] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a5b0afc8c84409981faf15563753388 in term 1.
I20260806 00:48:50.138307 18712 raft_consensus.cc:2454] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a5b0afc8c84409981faf15563753388 in term 1.
I20260806 00:48:50.138394 18827 leader_election.cc:302] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a5b0afc8c84409981faf15563753388, 944d83c0514b408482a985429baa67c4; no voters:
I20260806 00:48:50.138582 19368 raft_consensus.cc:2781] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:50.138691 19368 raft_consensus.cc:686] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [term 1 LEADER]: Becoming Leader. State: Replica: 4a5b0afc8c84409981faf15563753388, State: Running, Role: LEADER
I20260806 00:48:50.138855 19368 consensus_queue.cc:237] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:50.140674 18450 catalog_manager.cc:5495] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a5b0afc8c84409981faf15563753388 (127.15.147.130). New cstate: current_term: 1 leader_uuid: "4a5b0afc8c84409981faf15563753388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } health_report { overall_health: UNKNOWN } } }
I20260806 00:48:50.165949 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:50.162623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44058 (local address 127.15.147.130:46793)
0806 00:48:50.162892 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:50.162899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:50.162911 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:48:50.162950 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:50.162952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:50.162990 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:50.163213 (+ 223us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:50.163472 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.164036 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.164970 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.165184 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.165279 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:50.165535 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:50.165600 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:50.165654 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:50.165793 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 00:48:50.165832 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":110,"threads_started":1}
I20260806 00:48:50.171830 19161 raft_consensus.cc:1261] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4 [term 1 FOLLOWER]: Refusing update from remote peer 4a5b0afc8c84409981faf15563753388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:50.171895 18712 raft_consensus.cc:1261] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c [term 1 FOLLOWER]: Refusing update from remote peer 4a5b0afc8c84409981faf15563753388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:50.172290 19316 consensus_queue.cc:1035] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:50.172464 19368 consensus_queue.cc:1035] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:50.203660 18448 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:48:50.203835 18448 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:48:50.206015 18448 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0: Sending DeleteTablet for 3 replicas of tablet 98dd9fa222a048eaa6ded1a9ad818625
I20260806 00:48:50.206589 18909 tablet_service.cc:1514] Processing DeleteTablet for tablet 98dd9fa222a048eaa6ded1a9ad818625 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:48:50 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:44046
I20260806 00:48:50.206756 19385 tablet_replica.cc:330] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388: stopping tablet replica
I20260806 00:48:50.206725 18675 tablet_service.cc:1514] Processing DeleteTablet for tablet 98dd9fa222a048eaa6ded1a9ad818625 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:48:50 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:57812
I20260806 00:48:50.206869 19385 raft_consensus.cc:2229] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:48:50.206929 19384 tablet_replica.cc:330] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c: stopping tablet replica
I20260806 00:48:50.207029 19385 raft_consensus.cc:2258] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:48:50.207049 19384 raft_consensus.cc:2229] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:48:50.207181 19384 raft_consensus.cc:2258] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:48:50.207836 19385 ts_tablet_manager.cc:1905] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:48:50.207975 19384 ts_tablet_manager.cc:1905] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:48:50.208189 18448 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:48:50.208734 18448 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:48:50.210289 19385 ts_tablet_manager.cc:1918] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:48:50.210371 19385 log.cc:1196] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/98dd9fa222a048eaa6ded1a9ad818625
I20260806 00:48:50.210618 19385 ts_tablet_manager.cc:1939] T 98dd9fa222a048eaa6ded1a9ad818625 P 4a5b0afc8c84409981faf15563753388: Deleting consensus metadata
I20260806 00:48:50.210629 19384 ts_tablet_manager.cc:1918] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:48:50.210731 19384 log.cc:1196] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-0/wal/wals/98dd9fa222a048eaa6ded1a9ad818625
I20260806 00:48:50.210978 19384 ts_tablet_manager.cc:1939] T 98dd9fa222a048eaa6ded1a9ad818625 P 13b69f2f7f684c4bae51d211aee4ac9c: Deleting consensus metadata
I20260806 00:48:50.211292 18426 catalog_manager.cc:4841] TS 4a5b0afc8c84409981faf15563753388 (127.15.147.130:46793): tablet 98dd9fa222a048eaa6ded1a9ad818625 (table test-table [id=f0f7ba2628f048d1942f925ddb85b048]) successfully deleted
I20260806 00:48:50.211687 18429 catalog_manager.cc:4841] TS 13b69f2f7f684c4bae51d211aee4ac9c (127.15.147.129:40491): tablet 98dd9fa222a048eaa6ded1a9ad818625 (table test-table [id=f0f7ba2628f048d1942f925ddb85b048]) successfully deleted
I20260806 00:48:50.211799 19108 tablet_service.cc:1514] Processing DeleteTablet for tablet 98dd9fa222a048eaa6ded1a9ad818625 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:48:50 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:60416
I20260806 00:48:50.212013 19386 tablet_replica.cc:330] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4: stopping tablet replica
I20260806 00:48:50.212144 19386 raft_consensus.cc:2229] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:48:50.212245 19386 raft_consensus.cc:2258] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:48:50.213804 19386 ts_tablet_manager.cc:1905] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:48:50.216149 19386 ts_tablet_manager.cc:1918] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:48:50.216229 19386 log.cc:1196] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/98dd9fa222a048eaa6ded1a9ad818625
I20260806 00:48:50.216486 19386 ts_tablet_manager.cc:1939] T 98dd9fa222a048eaa6ded1a9ad818625 P 944d83c0514b408482a985429baa67c4: Deleting consensus metadata
I20260806 00:48:50.217113 18424 catalog_manager.cc:4841] TS 944d83c0514b408482a985429baa67c4 (127.15.147.131:44909): tablet 98dd9fa222a048eaa6ded1a9ad818625 (table test-table [id=f0f7ba2628f048d1942f925ddb85b048]) successfully deleted
I20260806 00:48:50.218302 18675 tablet_service.cc:1467] Processing CreateTablet for tablet 05a956e011284546b76232ba5ece28dc (DEFAULT_TABLE table=test-table [id=33a9ddf0dcb149f781a1968dc9cd5bb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:50.218441 18909 tablet_service.cc:1467] Processing CreateTablet for tablet 05a956e011284546b76232ba5ece28dc (DEFAULT_TABLE table=test-table [id=33a9ddf0dcb149f781a1968dc9cd5bb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:50.218597 18675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05a956e011284546b76232ba5ece28dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:50.218710 18909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05a956e011284546b76232ba5ece28dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:50.220288 19401 tablet_bootstrap.cc:492] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c: Bootstrap starting.
I20260806 00:48:50.220422 19405 tablet_bootstrap.cc:492] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388: Bootstrap starting.
I20260806 00:48:50.221001 19108 tablet_service.cc:1467] Processing CreateTablet for tablet 05a956e011284546b76232ba5ece28dc (DEFAULT_TABLE table=test-table [id=33a9ddf0dcb149f781a1968dc9cd5bb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:48:50.221230 19108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05a956e011284546b76232ba5ece28dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:50.221266 19401 tablet_bootstrap.cc:654] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:50.221271 19405 tablet_bootstrap.cc:654] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:50.222431 19401 tablet_bootstrap.cc:492] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c: No bootstrap required, opened a new log
I20260806 00:48:50.222528 19401 ts_tablet_manager.cc:1397] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 00:48:50.222932 19401 raft_consensus.cc:348] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:50.223054 19401 raft_consensus.cc:374] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:50.223100 19401 raft_consensus.cc:729] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13b69f2f7f684c4bae51d211aee4ac9c, State: Initialized, Role: FOLLOWER
I20260806 00:48:50.223233 19401 consensus_queue.cc:260] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:50.223315 19404 tablet_bootstrap.cc:492] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4: Bootstrap starting.
I20260806 00:48:50.223548 19401 ts_tablet_manager.cc:1428] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 00:48:50.224126 19404 tablet_bootstrap.cc:654] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:50.224251 19405 tablet_bootstrap.cc:492] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388: No bootstrap required, opened a new log
I20260806 00:48:50.224380 19405 ts_tablet_manager.cc:1397] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260806 00:48:50.224835 19405 raft_consensus.cc:348] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:50.224972 19405 raft_consensus.cc:374] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:50.225009 19405 raft_consensus.cc:729] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a5b0afc8c84409981faf15563753388, State: Initialized, Role: FOLLOWER
I20260806 00:48:50.225152 19405 consensus_queue.cc:260] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:50.225417 19404 tablet_bootstrap.cc:492] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4: No bootstrap required, opened a new log
I20260806 00:48:50.225422 19405 ts_tablet_manager.cc:1428] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:48:50.225508 19404 ts_tablet_manager.cc:1397] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 00:48:50.225924 19404 raft_consensus.cc:348] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:50.226070 19404 raft_consensus.cc:374] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:50.226125 19404 raft_consensus.cc:729] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 944d83c0514b408482a985429baa67c4, State: Initialized, Role: FOLLOWER
I20260806 00:48:50.226255 19404 consensus_queue.cc:260] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:50.226560 19404 ts_tablet_manager.cc:1428] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:48:50.553521 19410 raft_consensus.cc:482] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:48:50.553707 19410 raft_consensus.cc:504] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:50.555080 19410 leader_election.cc:288] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a5b0afc8c84409981faf15563753388 (127.15.147.130:46793), 944d83c0514b408482a985429baa67c4 (127.15.147.131:44909)
I20260806 00:48:50.560906 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:50.555428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:46523 (local address 127.15.147.130:46793)
0806 00:48:50.555638 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:50.555643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:50.555741 (+ 98us) server_negotiation.cc:407] Connection header received
0806 00:48:50.556018 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:50.556021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:50.556079 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:50.556166 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:50.556612 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.557430 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.559485 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.560113 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.560248 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:50.560336 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:50.560670 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 00:48:50.560754 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260806 00:48:50.560907 19427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:50.555528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:42595 (local address 127.15.147.131:44909)
0806 00:48:50.555821 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:50.555837 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:50.555854 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:48:50.556031 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:50.556034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:50.556079 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:50.556166 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:48:50.556612 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.557432 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.559486 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.560113 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.560248 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:50.560336 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:50.560670 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 00:48:50.560750 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":76,"threads_started":1}
I20260806 00:48:50.561004 19425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:50.555297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:46793 (local address 127.15.147.129:46523)
0806 00:48:50.555623 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:50.555639 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:50.555833 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:50.556204 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:50.556207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:50.556254 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:50.556484 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:50.556491 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.557579 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.557582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:50.559351 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:50.559363 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.560250 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.560254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:50.560469 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:50.560557 (+ 88us) client_negotiation.cc:769] Sending connection context
0806 00:48:50.560636 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 00:48:50.560813 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":375,"spinlock_wait_cycles":8192,"thread_start_us":93,"threads_started":1}
I20260806 00:48:50.561004 19426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:50.555429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:44909 (local address 127.15.147.129:42595)
0806 00:48:50.555792 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:50.555804 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:50.555891 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:50.556204 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:50.556207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:50.556253 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:48:50.556484 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:50.556491 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.557579 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.557582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:50.559351 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:50.559363 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.560250 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.560254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:50.560474 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:50.560556 (+ 82us) client_negotiation.cc:769] Sending connection context
0806 00:48:50.560636 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 00:48:50.560813 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":402,"spinlock_wait_cycles":32384,"thread_start_us":132,"threads_started":1}
I20260806 00:48:50.561659 19161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05a956e011284546b76232ba5ece28dc" candidate_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "944d83c0514b408482a985429baa67c4" is_pre_election: true
I20260806 00:48:50.561775 18942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05a956e011284546b76232ba5ece28dc" candidate_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a5b0afc8c84409981faf15563753388" is_pre_election: true
I20260806 00:48:50.561856 19161 raft_consensus.cc:2454] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13b69f2f7f684c4bae51d211aee4ac9c in term 0.
I20260806 00:48:50.561926 18942 raft_consensus.cc:2454] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13b69f2f7f684c4bae51d211aee4ac9c in term 0.
I20260806 00:48:50.562309 18597 leader_election.cc:302] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 13b69f2f7f684c4bae51d211aee4ac9c, 4a5b0afc8c84409981faf15563753388, 944d83c0514b408482a985429baa67c4; no voters:
I20260806 00:48:50.562588 19410 raft_consensus.cc:2781] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:48:50.562645 19410 raft_consensus.cc:482] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:48:50.562671 19410 raft_consensus.cc:3037] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:50.564234 19410 raft_consensus.cc:504] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:50.564663 19410 leader_election.cc:288] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [CANDIDATE]: Term 1 election: Requested vote from peers 4a5b0afc8c84409981faf15563753388 (127.15.147.130:46793), 944d83c0514b408482a985429baa67c4 (127.15.147.131:44909)
I20260806 00:48:50.565008 19161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05a956e011284546b76232ba5ece28dc" candidate_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "944d83c0514b408482a985429baa67c4"
I20260806 00:48:50.565008 18942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05a956e011284546b76232ba5ece28dc" candidate_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a5b0afc8c84409981faf15563753388"
I20260806 00:48:50.565151 18942 raft_consensus.cc:3037] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:50.565151 19161 raft_consensus.cc:3037] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:50.566322 18942 raft_consensus.cc:2454] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13b69f2f7f684c4bae51d211aee4ac9c in term 1.
I20260806 00:48:50.566322 19161 raft_consensus.cc:2454] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13b69f2f7f684c4bae51d211aee4ac9c in term 1.
I20260806 00:48:50.566699 18598 leader_election.cc:302] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 13b69f2f7f684c4bae51d211aee4ac9c, 4a5b0afc8c84409981faf15563753388, 944d83c0514b408482a985429baa67c4; no voters:
I20260806 00:48:50.566900 19410 raft_consensus.cc:2781] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:50.567107 19410 raft_consensus.cc:686] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [term 1 LEADER]: Becoming Leader. State: Replica: 13b69f2f7f684c4bae51d211aee4ac9c, State: Running, Role: LEADER
I20260806 00:48:50.567385 19410 consensus_queue.cc:237] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } }
I20260806 00:48:50.570009 18450 catalog_manager.cc:5495] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c reported cstate change: term changed from 0 to 1, leader changed from <none> to 13b69f2f7f684c4bae51d211aee4ac9c (127.15.147.129). New cstate: current_term: 1 leader_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b69f2f7f684c4bae51d211aee4ac9c" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 } health_report { overall_health: UNKNOWN } } }
I20260806 00:48:50.592059 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:50.588175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57816 (local address 127.15.147.129:40491)
0806 00:48:50.588528 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:50.588535 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:50.588545 (+ 10us) server_negotiation.cc:407] Connection header received
0806 00:48:50.588612 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:50.588614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:50.588664 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:50.588802 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:50.589087 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.589703 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.590535 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.590790 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.590925 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:50.591195 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:50.591655 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:50.591789 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:50.591828 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 00:48:50.591884 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":177,"threads_started":1}
I20260806 00:48:50.602679 19427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:50.599235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:47563 (local address 127.15.147.131:44909)
0806 00:48:50.599399 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:50.599405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:50.599535 (+ 130us) server_negotiation.cc:407] Connection header received
0806 00:48:50.599700 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:50.599703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:50.599758 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:50.599844 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:50.600148 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.600660 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.601524 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.601778 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.601898 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:50.602224 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:50.602297 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:50.602383 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:50.602478 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 00:48:50.602532 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:48:50.602698 19434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:50.599070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:44909 (local address 127.15.147.129:47563)
0806 00:48:50.599450 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:50.599463 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:50.599593 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:50.599878 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:50.599881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:50.599891 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:48:50.600018 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:50.600023 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.600781 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.600785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:50.601395 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:50.601403 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.601922 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.601926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:50.602021 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:50.602065 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:48:50.602400 (+ 335us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:48:50.602407 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 00:48:50.602468 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 00:48:50.602551 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":106,"threads_started":1}
I20260806 00:48:50.608600 18942 raft_consensus.cc:1261] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388 [term 1 FOLLOWER]: Refusing update from remote peer 13b69f2f7f684c4bae51d211aee4ac9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:50.608608 19161 raft_consensus.cc:1261] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4 [term 1 FOLLOWER]: Refusing update from remote peer 13b69f2f7f684c4bae51d211aee4ac9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:48:50.609159 19429 consensus_queue.cc:1035] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "944d83c0514b408482a985429baa67c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:50.609364 19410 consensus_queue.cc:1035] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a5b0afc8c84409981faf15563753388" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:48:50.626461 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:50.622915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:55869 (local address 127.15.147.129:40491)
0806 00:48:50.623035 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:50.623039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:50.623239 (+ 200us) server_negotiation.cc:407] Connection header received
0806 00:48:50.623356 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:50.623359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:50.623408 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:50.623466 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:48:50.623762 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.624274 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.625283 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.625612 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.625731 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:50.626011 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:48:50.626113 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:48:50.626179 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:50.626276 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 00:48:50.626328 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 00:48:50.626578 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:50.622864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:40491 (local address 127.15.147.131:55869)
0806 00:48:50.623137 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:50.623152 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:50.623258 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:50.623492 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:50.623497 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:50.623510 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:48:50.623632 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:50.623641 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.624419 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.624423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:50.625137 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:50.625149 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:50.625744 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:50.625747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:50.625846 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:50.625878 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:48:50.626184 (+ 306us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:48:50.626193 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 00:48:50.626257 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 00:48:50.626351 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":111,"threads_started":1}
W20260806 00:48:50.627224 19357 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 00:48:52.185946 18448 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:48:52.186095 18448 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:48:52.188434 18448 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0729b77006743698f3584e1fea9a1c0: Sending DeleteTablet for 3 replicas of tablet 05a956e011284546b76232ba5ece28dc
I20260806 00:48:52.189042 18675 tablet_service.cc:1514] Processing DeleteTablet for tablet 05a956e011284546b76232ba5ece28dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:48:52 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:57812
I20260806 00:48:52.189070 18909 tablet_service.cc:1514] Processing DeleteTablet for tablet 05a956e011284546b76232ba5ece28dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:48:52 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:44046
I20260806 00:48:52.189442 19475 tablet_replica.cc:330] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388: stopping tablet replica
I20260806 00:48:52.189420 19108 tablet_service.cc:1514] Processing DeleteTablet for tablet 05a956e011284546b76232ba5ece28dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:48:52 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:60416
I20260806 00:48:52.189456 19474 tablet_replica.cc:330] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c: stopping tablet replica
I20260806 00:48:52.189579 19474 raft_consensus.cc:2229] T 05a956e011284546b76232ba5ece28dc P 13b69f2f7f684c4bae51d211aee4ac9c [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:48:52.189594 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 18569
I20260806 00:48:52.196144 19475 raft_consensus.cc:2229] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:48:52.196377 19475 raft_consensus.cc:2258] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:48:52.205199 19475 ts_tablet_manager.cc:1905] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:48:52.206099 19476 tablet_replica.cc:330] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4: stopping tablet replica
I20260806 00:48:52.206293 19476 raft_consensus.cc:2229] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:48:52.206475 19476 raft_consensus.cc:2258] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:48:52.207212 19476 ts_tablet_manager.cc:1905] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:48:52.207890 19475 ts_tablet_manager.cc:1918] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 00:48:52.207968 19475 log.cc:1196] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/05a956e011284546b76232ba5ece28dc
I20260806 00:48:52.208230 19475 ts_tablet_manager.cc:1939] T 05a956e011284546b76232ba5ece28dc P 4a5b0afc8c84409981faf15563753388: Deleting consensus metadata
I20260806 00:48:52.208937 18426 catalog_manager.cc:4841] TS 4a5b0afc8c84409981faf15563753388 (127.15.147.130:46793): tablet 05a956e011284546b76232ba5ece28dc (table test-table [id=33a9ddf0dcb149f781a1968dc9cd5bb3]) successfully deleted
I20260806 00:48:52.210314 19476 ts_tablet_manager.cc:1918] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 00:48:52.210459 19476 log.cc:1196] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/05a956e011284546b76232ba5ece28dc
W20260806 00:48:52.210456 18429 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.147.129:40491 (error 108)
I20260806 00:48:52.210718 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 18808
I20260806 00:48:52.213685 19476 ts_tablet_manager.cc:1939] T 05a956e011284546b76232ba5ece28dc P 944d83c0514b408482a985429baa67c4: Deleting consensus metadata
I20260806 00:48:52.214617 18424 catalog_manager.cc:4841] TS 944d83c0514b408482a985429baa67c4 (127.15.147.131:44909): tablet 05a956e011284546b76232ba5ece28dc (table test-table [id=33a9ddf0dcb149f781a1968dc9cd5bb3]) successfully deleted
I20260806 00:48:52.214986 19479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:52.213838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:40491 (local address 127.0.0.1:57832)
0806 00:48:52.214806 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:52.214893 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:40491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":87,"threads_started":1}
W20260806 00:48:52.215080 18429 catalog_manager.cc:4584] TS 13b69f2f7f684c4bae51d211aee4ac9c (127.15.147.129:40491): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 05a956e011284546b76232ba5ece28dc: Network error: Client connection negotiation failed: client connection to 127.15.147.129:40491: connect: Connection refused (error 111)
I20260806 00:48:52.223312 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 19038
I20260806 00:48:52.238487 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 18404
2026-08-06T00:48:52Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6655 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:52 dist-test-slave-45dx krb5kdc[19484](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:52 dist-test-slave-45dx krb5kdc[19484](info): set up 2 sockets
Aug 06 00:48:52 dist-test-slave-45dx krb5kdc[19484](info): commencing operation
krb5kdc: starting...
W20260806 00:48:53.452836 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:48:53 dist-test-slave-45dx krb5kdc[19484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:48:53.577299 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:37131
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:37131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:53.688525 19505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:53.688823 19505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:53.688867 19505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:53.693018 19505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:53.693078 19505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:53.693097 19505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:53.693114 19505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:53.693166 19505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:53.698478 19505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:37131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:37131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:53.699620 19505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:53.700726 19505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:53.709537 19511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:53.709549 19516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:53.709613 19505 server_base.cc:1034] running on GCE node
W20260806 00:48:53.709560 19513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:53.710379 19505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:53.711454 19505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:53.712612 19505 hybrid_clock.cc:648] HybridClock initialized: now 1785977333712583 us; error 39 us; skew 500 ppm
Aug 06 00:48:53 dist-test-slave-45dx krb5kdc[19484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977333, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:53.718632 19505 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:48:53.719439 19505 webserver.cc:460] Webserver started at http://127.15.147.190:35813/ using document root <none> and password file <none>
I20260806 00:48:53.719962 19505 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:53.720021 19505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:53.720182 19505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:53.722249 19505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "3fa1554544f847a48f52f184c3e46368"
format_stamp: "Formatted at 2026-08-06 00:48:53 on dist-test-slave-45dx"
server_key: "9eee99b4a5fe725a18a9f0dc42955a44"
server_key_iv: "060e4a44671c24f42ca56835be956cab"
server_key_version: "encryptionkey@0"
I20260806 00:48:53.722770 19505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "3fa1554544f847a48f52f184c3e46368"
format_stamp: "Formatted at 2026-08-06 00:48:53 on dist-test-slave-45dx"
server_key: "9eee99b4a5fe725a18a9f0dc42955a44"
server_key_iv: "060e4a44671c24f42ca56835be956cab"
server_key_version: "encryptionkey@0"
I20260806 00:48:53.726387 19505 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 00:48:53.728679 19526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:53.729882 19505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:48:53.729977 19505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "3fa1554544f847a48f52f184c3e46368"
format_stamp: "Formatted at 2026-08-06 00:48:53 on dist-test-slave-45dx"
server_key: "9eee99b4a5fe725a18a9f0dc42955a44"
server_key_iv: "060e4a44671c24f42ca56835be956cab"
server_key_version: "encryptionkey@0"
I20260806 00:48:53.730055 19505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:53.741382 19505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:53.744067 19505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:53.744210 19505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:53.750952 19505 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:37131
I20260806 00:48:53.751035 19610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:37131 every 8 connection(s)
I20260806 00:48:53.751904 19505 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:53.753710 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 19505
I20260806 00:48:53.753823 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:48:53.754114 15950 external_mini_cluster.cc:1428] Setting key b4c4b39e8fd458703283daf668bf706e
I20260806 00:48:53.755059 19612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:53.761267 19612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368: Bootstrap starting.
Aug 06 00:48:53 dist-test-slave-45dx krb5kdc[19484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:53.763890 19612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:53.764863 19612 log.cc:824] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:53.767058 19612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368: No bootstrap required, opened a new log
I20260806 00:48:53.768368 19615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:53.755450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43336 (local address 127.15.147.190:37131)
0806 00:48:53.755834 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:53.755849 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:53.755881 (+ 32us) server_negotiation.cc:407] Connection header received
0806 00:48:53.756575 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:53.756591 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:53.756880 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:53.757135 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:53.758049 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:53.758947 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:53.760114 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:53.760606 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:53.760715 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:53.763286 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:53.763293 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:53.763307 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:53.763337 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:53.765359 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:53.765831 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:53.765836 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:53.765841 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:53.765991 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:53.766316 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:53.766319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:53.766321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:53.766793 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:53.766974 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:53.767193 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 00:48:53.767480 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":129,"threads_started":1}
I20260806 00:48:53.771186 19612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fa1554544f847a48f52f184c3e46368" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37131 } }
I20260806 00:48:53.771400 19612 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:53.771436 19612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fa1554544f847a48f52f184c3e46368, State: Initialized, Role: FOLLOWER
I20260806 00:48:53.771763 19612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fa1554544f847a48f52f184c3e46368" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37131 } }
I20260806 00:48:53.771867 19612 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:53.771910 19612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:53.771976 19612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:53.772812 19612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fa1554544f847a48f52f184c3e46368" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37131 } }
I20260806 00:48:53.773154 19612 leader_election.cc:302] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fa1554544f847a48f52f184c3e46368; no voters:
I20260806 00:48:53.773479 19612 leader_election.cc:288] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:53.773587 19617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:53.773770 19617 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368 [term 1 LEADER]: Becoming Leader. State: Replica: 3fa1554544f847a48f52f184c3e46368, State: Running, Role: LEADER
I20260806 00:48:53.774062 19617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fa1554544f847a48f52f184c3e46368" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37131 } }
I20260806 00:48:53.774701 19612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:53.775799 19620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fa1554544f847a48f52f184c3e46368. Latest consensus state: current_term: 1 leader_uuid: "3fa1554544f847a48f52f184c3e46368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fa1554544f847a48f52f184c3e46368" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37131 } } }
I20260806 00:48:53.775837 19619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fa1554544f847a48f52f184c3e46368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fa1554544f847a48f52f184c3e46368" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37131 } } }
I20260806 00:48:53.775934 19620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:53.775942 19619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:53.776401 19630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:53.779520 19630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:53.784420 19630 catalog_manager.cc:1347] Generated new cluster ID: 1f0ccbe9743c4767a5f69caa7d04e0ac
I20260806 00:48:53.784482 19630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:53.793323 19630 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:53.794399 19630 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:53.797817 19630 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fa1554544f847a48f52f184c3e46368: Generated new TSK 0
I20260806 00:48:53.798410 19630 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
I20260806 00:48:53.832883 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:37131
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:53.949606 19645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:53.949901 19645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:53.949949 19645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:53.954155 19645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:53.954232 19645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:53.954331 19645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:48:53.960022 19645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:53.961298 19645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:53.962411 19645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:53.972218 19654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:53.972205 19656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:53.972239 19645 server_base.cc:1034] running on GCE node
W20260806 00:48:53.972201 19658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:53.973006 19645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:53.973621 19645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:53.974850 19645 hybrid_clock.cc:648] HybridClock initialized: now 1785977333974813 us; error 53 us; skew 500 ppm
Aug 06 00:48:53 dist-test-slave-45dx krb5kdc[19484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977333, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:53.977772 19645 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:48:53.978655 19645 webserver.cc:460] Webserver started at http://127.15.147.129:36763/ using document root <none> and password file <none>
I20260806 00:48:53.979200 19645 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:53.979286 19645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:53.979504 19645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:53.981545 19645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "4f62b75554a940118f0c8fd6532faa57"
format_stamp: "Formatted at 2026-08-06 00:48:53 on dist-test-slave-45dx"
server_key: "e75a7c2de145f6cc920b816048500790"
server_key_iv: "10ade0556c92ceeb893871a3d30ebaa2"
server_key_version: "encryptionkey@0"
I20260806 00:48:53.982154 19645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "4f62b75554a940118f0c8fd6532faa57"
format_stamp: "Formatted at 2026-08-06 00:48:53 on dist-test-slave-45dx"
server_key: "e75a7c2de145f6cc920b816048500790"
server_key_iv: "10ade0556c92ceeb893871a3d30ebaa2"
server_key_version: "encryptionkey@0"
I20260806 00:48:53.985778 19645 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:48:53.988106 19668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:53.989478 19645 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:48:53.989622 19645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "4f62b75554a940118f0c8fd6532faa57"
format_stamp: "Formatted at 2026-08-06 00:48:53 on dist-test-slave-45dx"
server_key: "e75a7c2de145f6cc920b816048500790"
server_key_iv: "10ade0556c92ceeb893871a3d30ebaa2"
server_key_version: "encryptionkey@0"
I20260806 00:48:53.989737 19645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:54.000903 19645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:54.003836 19645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:54.004042 19645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:54.004686 19645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:48:54.005515 19645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:48:54.005600 19645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:54.005692 19645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:48:54.005743 19645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:54.015189 19645 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:42775
I20260806 00:48:54.015213 19835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:42775 every 8 connection(s)
I20260806 00:48:54.016227 19645 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:48:54.019919 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 19645
I20260806 00:48:54.020016 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:48:54.020364 15950 external_mini_cluster.cc:1428] Setting key cd705607cb6fdce6b821ab4a627a2dba
Aug 06 00:48:54 dist-test-slave-45dx krb5kdc[19484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977333, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.147.130@KRBTEST.COM; defaulting to no policy
I20260806 00:48:54.032248 19615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:54.018219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:39981 (local address 127.15.147.190:37131)
0806 00:48:54.018399 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:54.018403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:54.019283 (+ 880us) server_negotiation.cc:407] Connection header received
0806 00:48:54.020909 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:54.020912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:54.020992 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:54.021101 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:54.022623 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:54.023469 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:54.024773 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:54.025185 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:54.025308 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:54.028239 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:54.028243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:54.028245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:54.028271 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:54.029882 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:54.030549 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:54.030552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:54.030553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:54.030635 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:54.031030 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:54.031032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:54.031033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:54.031209 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:54.031286 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:54.031970 (+ 684us) server_negotiation.cc:299] Negotiation successful
0806 00:48:54.032081 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 00:48:54.033366 19840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:54.018611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37131 (local address 127.15.147.129:39981)
0806 00:48:54.019110 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:54.019156 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:54.020640 (+ 1484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:54.021333 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:54.021342 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:54.021897 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:54.022468 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:54.022483 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:54.023614 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:54.023617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:54.024644 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:54.024655 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:54.025353 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:54.025356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:54.025496 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:54.026216 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:54.026234 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:54.028038 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:54.030011 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:54.030018 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:54.030034 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:54.030436 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:54.030727 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:54.030732 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:54.030734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:54.030926 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:54.031325 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:54.031334 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:54.031695 (+ 361us) client_negotiation.cc:769] Sending connection context
0806 00:48:54.031926 (+ 231us) client_negotiation.cc:240] Negotiation successful
0806 00:48:54.032327 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":138,"threads_started":1}
I20260806 00:48:54.034641 19836 heartbeater.cc:347] Connected to a master server at 127.15.147.190:37131
I20260806 00:48:54.034876 19836 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.130@KRBTEST.COM" created.
I20260806 00:48:54.035463 19836 heartbeater.cc:511] Master 127.15.147.190:37131 requested a full tablet report, sending...
I20260806 00:48:54.037154 19553 ts_manager.cc:194] Registered new tserver with Master: 4f62b75554a940118f0c8fd6532faa57 (127.15.147.129:42775)
I20260806 00:48:54.038321 19553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:39981
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
I20260806 00:48:54.051771 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:37131
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:54.167471 19846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:54.167716 19846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:54.167758 19846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:54.171984 19846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:54.172041 19846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:54.172144 19846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:48:54.177558 19846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:54.178503 19846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:54.179534 19846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:54.189669 19858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:54.189669 19854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:54.189708 19846 server_base.cc:1034] running on GCE node
W20260806 00:48:54.189658 19855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:54.190356 19846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:54.190917 19846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:54.192088 19846 hybrid_clock.cc:648] HybridClock initialized: now 1785977334192072 us; error 27 us; skew 500 ppm
Aug 06 00:48:54 dist-test-slave-45dx krb5kdc[19484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977334, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:54.195030 19846 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:48:54.195881 19846 webserver.cc:460] Webserver started at http://127.15.147.130:46061/ using document root <none> and password file <none>
I20260806 00:48:54.196430 19846 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:54.196496 19846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:54.196661 19846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:54.198572 19846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "16b2e9d484fd4904829d1f190d4758bf"
format_stamp: "Formatted at 2026-08-06 00:48:54 on dist-test-slave-45dx"
server_key: "d6f513f64595a768f8dc21de483df3c5"
server_key_iv: "5a42b85f912a6ed9fa5ec0b6f154d9d8"
server_key_version: "encryptionkey@0"
I20260806 00:48:54.199131 19846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "16b2e9d484fd4904829d1f190d4758bf"
format_stamp: "Formatted at 2026-08-06 00:48:54 on dist-test-slave-45dx"
server_key: "d6f513f64595a768f8dc21de483df3c5"
server_key_iv: "5a42b85f912a6ed9fa5ec0b6f154d9d8"
server_key_version: "encryptionkey@0"
I20260806 00:48:54.202773 19846 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:48:54.205277 19868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:54.206508 19846 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:48:54.206611 19846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "16b2e9d484fd4904829d1f190d4758bf"
format_stamp: "Formatted at 2026-08-06 00:48:54 on dist-test-slave-45dx"
server_key: "d6f513f64595a768f8dc21de483df3c5"
server_key_iv: "5a42b85f912a6ed9fa5ec0b6f154d9d8"
server_key_version: "encryptionkey@0"
I20260806 00:48:54.206770 19846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:54.218479 19846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:54.221562 19846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:54.221791 19846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:54.222409 19846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:48:54.223263 19846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:48:54.223341 19846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:54.223413 19846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:48:54.223466 19846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:54.233110 19846 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:43209
I20260806 00:48:54.233129 20046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:43209 every 8 connection(s)
I20260806 00:48:54.234086 19846 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:48:54.239892 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 19846
I20260806 00:48:54.240033 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:48:54.240401 15950 external_mini_cluster.cc:1428] Setting key fcdf39dc6fbf8d42d2f60bf46217d9ef
Aug 06 00:48:54 dist-test-slave-45dx krb5kdc[19484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977334, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:54.249095 19615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:54.235956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:38663 (local address 127.15.147.190:37131)
0806 00:48:54.236099 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:54.236103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:54.236977 (+ 874us) server_negotiation.cc:407] Connection header received
0806 00:48:54.238229 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:54.238232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:54.238279 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:54.238356 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:54.239455 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:54.240048 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:54.241662 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:54.242017 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:54.242114 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:54.245091 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:54.245097 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:54.245101 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:54.245134 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:54.246654 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:54.247281 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:54.247283 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:54.247285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:54.247358 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:54.247838 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:54.247844 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:54.247846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:54.248058 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:54.248136 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:54.248867 (+ 731us) server_negotiation.cc:299] Negotiation successful
0806 00:48:54.248970 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:48:54.250265 20050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:54.236374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37131 (local address 127.15.147.130:38663)
0806 00:48:54.236828 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:54.236870 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:54.238030 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:54.238516 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:54.238522 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:54.238870 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:54.239328 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:54.239341 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:54.240180 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:54.240184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:54.241512 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:54.241523 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:54.242191 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:54.242196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:54.242407 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:54.243102 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:54.243129 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:54.244916 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:54.246780 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:54.246789 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:54.246804 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:54.247176 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:54.247462 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:54.247468 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:54.247473 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:54.247686 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:54.248177 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:54.248183 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:54.248537 (+ 354us) client_negotiation.cc:769] Sending connection context
0806 00:48:54.248757 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 00:48:54.249129 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":119,"threads_started":1}
I20260806 00:48:54.251621 20047 heartbeater.cc:347] Connected to a master server at 127.15.147.190:37131
I20260806 00:48:54.251884 20047 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.147.131@KRBTEST.COM; defaulting to no policy
I20260806 00:48:54.252568 20047 heartbeater.cc:511] Master 127.15.147.190:37131 requested a full tablet report, sending...
I20260806 00:48:54.253625 19553 ts_manager.cc:194] Registered new tserver with Master: 16b2e9d484fd4904829d1f190d4758bf (127.15.147.130:43209)
I20260806 00:48:54.254206 19553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:38663
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
I20260806 00:48:54.271725 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:37131
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:54.388294 20053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:54.388559 20053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:54.388602 20053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:54.392567 20053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:54.392657 20053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:54.392763 20053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:48:54.398244 20053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:54.399194 20053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:54.400133 20053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:54.410000 20063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:54.410159 20053 server_base.cc:1034] running on GCE node
W20260806 00:48:54.410243 20062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:54.409999 20065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:54.411021 20053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:54.411662 20053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:54.412870 20053 hybrid_clock.cc:648] HybridClock initialized: now 1785977334412871 us; error 39 us; skew 500 ppm
Aug 06 00:48:54 dist-test-slave-45dx krb5kdc[19484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977334, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:54.416141 20053 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:48:54.417075 20053 webserver.cc:460] Webserver started at http://127.15.147.131:35815/ using document root <none> and password file <none>
I20260806 00:48:54.417647 20053 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:54.417737 20053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:54.417980 20053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:54.420094 20053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "581b179c290f4b4583b40536505947f2"
format_stamp: "Formatted at 2026-08-06 00:48:54 on dist-test-slave-45dx"
server_key: "d20858fff0d7cf25fddf6e7f441699a4"
server_key_iv: "d986591892d0f05d264da35091331ee6"
server_key_version: "encryptionkey@0"
I20260806 00:48:54.420724 20053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "581b179c290f4b4583b40536505947f2"
format_stamp: "Formatted at 2026-08-06 00:48:54 on dist-test-slave-45dx"
server_key: "d20858fff0d7cf25fddf6e7f441699a4"
server_key_iv: "d986591892d0f05d264da35091331ee6"
server_key_version: "encryptionkey@0"
I20260806 00:48:54.424582 20053 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 00:48:54.427049 20076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:54.428259 20053 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:48:54.428427 20053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "581b179c290f4b4583b40536505947f2"
format_stamp: "Formatted at 2026-08-06 00:48:54 on dist-test-slave-45dx"
server_key: "d20858fff0d7cf25fddf6e7f441699a4"
server_key_iv: "d986591892d0f05d264da35091331ee6"
server_key_version: "encryptionkey@0"
I20260806 00:48:54.428521 20053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:54.462527 20053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:54.465842 20053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:54.466015 20053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:54.466624 20053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:48:54.467502 20053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:48:54.467550 20053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:54.467593 20053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:48:54.467609 20053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:54.477813 20053 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:46423
I20260806 00:48:54.477849 20242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:46423 every 8 connection(s)
I20260806 00:48:54.478711 20053 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:48:54.479010 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 20053
I20260806 00:48:54.479135 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:48:54.479485 15950 external_mini_cluster.cc:1428] Setting key f82272d5dafde50fd7f544556e3cb38e
Aug 06 00:48:54 dist-test-slave-45dx krb5kdc[19484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977334, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:54.492743 19615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:54.480545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:34713 (local address 127.15.147.190:37131)
0806 00:48:54.480658 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:54.480662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:54.481793 (+ 1131us) server_negotiation.cc:407] Connection header received
0806 00:48:54.483190 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:54.483193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:54.483239 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:54.483311 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:54.484550 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:54.485074 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:54.486161 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:54.486380 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:54.486457 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:54.489034 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:54.489037 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:54.489039 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:54.489065 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:54.490602 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:54.491132 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:54.491135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:54.491136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:54.491217 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:54.491543 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:54.491545 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:54.491546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:54.491724 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:54.491801 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:54.492428 (+ 627us) server_negotiation.cc:299] Negotiation successful
0806 00:48:54.492586 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 00:48:54.493814 20246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:54.481096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37131 (local address 127.15.147.131:34713)
0806 00:48:54.481613 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:54.481677 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:54.482996 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:54.483482 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:54.483488 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:54.483860 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:54.484411 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:54.484436 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:54.485183 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:54.485186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:54.486030 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:54.486044 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:54.486510 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:54.486513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:54.486647 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:54.487251 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:54.487268 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:54.488880 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:54.490737 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:54.490742 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:54.490753 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:54.491042 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:54.491323 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:54.491325 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:54.491327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:54.491461 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:54.491850 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:54.491855 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:54.492090 (+ 235us) client_negotiation.cc:769] Sending connection context
0806 00:48:54.492336 (+ 246us) client_negotiation.cc:240] Negotiation successful
0806 00:48:54.492691 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":170,"threads_started":1}
I20260806 00:48:54.494961 20243 heartbeater.cc:347] Connected to a master server at 127.15.147.190:37131
I20260806 00:48:54.495210 20243 heartbeater.cc:464] Registering TS with master...
I20260806 00:48:54.495735 20243 heartbeater.cc:511] Master 127.15.147.190:37131 requested a full tablet report, sending...
I20260806 00:48:54.496852 19553 ts_manager.cc:194] Registered new tserver with Master: 581b179c290f4b4583b40536505947f2 (127.15.147.131:46423)
I20260806 00:48:54.497404 19553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:34713
I20260806 00:48:54.505811 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 00:48:54.520972 19615 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:48:54.518939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43338 (local address 127.15.147.190:37131)
0806 00:48:54.519097 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:54.519102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:54.519395 (+ 293us) server_negotiation.cc:407] Connection header received
0806 00:48:54.519745 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:54.519749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:54.519807 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:54.519904 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:54.520787 (+ 883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43338: BlockingRecv error: recv got EOF from 127.0.0.1:43338 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:48:54.522884 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 19645
I20260806 00:48:54.533944 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 19846
I20260806 00:48:54.556278 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 20053
I20260806 00:48:54.567638 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 19505
2026-08-06T00:48:54Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2326 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:54 dist-test-slave-45dx krb5kdc[20259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:54 dist-test-slave-45dx krb5kdc[20259](info): set up 2 sockets
Aug 06 00:48:54 dist-test-slave-45dx krb5kdc[20259](info): commencing operation
krb5kdc: starting...
W20260806 00:48:55.795579 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:48:55 dist-test-slave-45dx krb5kdc[20259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:48:55.923889 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:34967
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:45417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:34967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:56.037919 20276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:56.038182 20276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:56.038221 20276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:56.042376 20276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:56.042433 20276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:56.042452 20276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:56.042470 20276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:56.042485 20276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:56.047969 20276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:45417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:34967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:34967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:56.049062 20276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:56.050074 20276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:56.058650 20288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:56.058709 20289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:56.058836 20276 server_base.cc:1034] running on GCE node
W20260806 00:48:56.058652 20293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:56.059597 20276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:56.060731 20276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:56.061920 20276 hybrid_clock.cc:648] HybridClock initialized: now 1785977336061903 us; error 28 us; skew 500 ppm
Aug 06 00:48:56 dist-test-slave-45dx krb5kdc[20259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977336, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:56.064640 20276 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:48:56.065479 20276 webserver.cc:460] Webserver started at http://127.15.147.190:42283/ using document root <none> and password file <none>
I20260806 00:48:56.066006 20276 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:56.066097 20276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:56.066327 20276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:56.068293 20276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "37902f6912554cea99f84e037580ff2c"
format_stamp: "Formatted at 2026-08-06 00:48:56 on dist-test-slave-45dx"
server_key: "d11c611af44fd43e03520f6e45cdb073"
server_key_iv: "f132351f7b867a91ace9435134b5930a"
server_key_version: "encryptionkey@0"
I20260806 00:48:56.068938 20276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "37902f6912554cea99f84e037580ff2c"
format_stamp: "Formatted at 2026-08-06 00:48:56 on dist-test-slave-45dx"
server_key: "d11c611af44fd43e03520f6e45cdb073"
server_key_iv: "f132351f7b867a91ace9435134b5930a"
server_key_version: "encryptionkey@0"
I20260806 00:48:56.072760 20276 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:48:56.075125 20304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:56.076495 20276 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:48:56.076635 20276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "37902f6912554cea99f84e037580ff2c"
format_stamp: "Formatted at 2026-08-06 00:48:56 on dist-test-slave-45dx"
server_key: "d11c611af44fd43e03520f6e45cdb073"
server_key_iv: "f132351f7b867a91ace9435134b5930a"
server_key_version: "encryptionkey@0"
I20260806 00:48:56.076756 20276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:56.100370 20276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:56.103420 20276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:56.103643 20276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:56.111407 20276 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:34967
I20260806 00:48:56.111817 20381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:34967 every 8 connection(s)
I20260806 00:48:56.112583 20276 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:56.115483 20383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:56.120613 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 20276
I20260806 00:48:56.120738 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:48:56.120990 15950 external_mini_cluster.cc:1428] Setting key fb364b30de65fe14297825446fe79a59
I20260806 00:48:56.121173 20383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c: Bootstrap starting.
I20260806 00:48:56.123831 20383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:56.124754 20383 log.cc:824] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:56.126927 20383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c: No bootstrap required, opened a new log
Aug 06 00:48:56 dist-test-slave-45dx krb5kdc[20259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:56.131392 20383 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37902f6912554cea99f84e037580ff2c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34967 } }
I20260806 00:48:56.131682 20383 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:56.131775 20383 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37902f6912554cea99f84e037580ff2c, State: Initialized, Role: FOLLOWER
I20260806 00:48:56.132351 20383 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37902f6912554cea99f84e037580ff2c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34967 } }
I20260806 00:48:56.132539 20383 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:56.132622 20383 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:56.132735 20383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:56.133868 20383 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37902f6912554cea99f84e037580ff2c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34967 } }
I20260806 00:48:56.134224 20383 leader_election.cc:302] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37902f6912554cea99f84e037580ff2c; no voters:
I20260806 00:48:56.134557 20383 leader_election.cc:288] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:56.134656 20389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:56.134925 20389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c [term 1 LEADER]: Becoming Leader. State: Replica: 37902f6912554cea99f84e037580ff2c, State: Running, Role: LEADER
I20260806 00:48:56.135236 20389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37902f6912554cea99f84e037580ff2c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34967 } }
I20260806 00:48:56.135859 20383 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:56.136482 20389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37902f6912554cea99f84e037580ff2c. Latest consensus state: current_term: 1 leader_uuid: "37902f6912554cea99f84e037580ff2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37902f6912554cea99f84e037580ff2c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34967 } } }
I20260806 00:48:56.136620 20389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:56.136962 20394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37902f6912554cea99f84e037580ff2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37902f6912554cea99f84e037580ff2c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34967 } } }
I20260806 00:48:56.137061 20394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:56.137068 20401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:56.137396 20387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:56.122348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32920 (local address 127.15.147.190:34967)
0806 00:48:56.122779 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:56.122798 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:56.122839 (+ 41us) server_negotiation.cc:407] Connection header received
0806 00:48:56.123630 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:56.123655 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:56.124092 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:56.124486 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:56.125239 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:56.126667 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:56.127602 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:56.128240 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:56.128440 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:56.131233 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:56.131239 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:56.131254 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:56.131283 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:56.133479 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:56.133918 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:56.133924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:56.133930 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:56.134070 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:56.134371 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:56.134374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:56.134376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:56.134926 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:56.135696 (+ 770us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:56.135859 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 00:48:56.136207 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":110,"threads_started":1}
I20260806 00:48:56.140164 20401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:56.140936 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:48:56.147042 20401 catalog_manager.cc:1347] Generated new cluster ID: 4277c35254124aa2b93542f15e7f7817
I20260806 00:48:56.147107 20401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:56.163782 20387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:56.158686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32932 (local address 127.15.147.190:34967)
0806 00:48:56.158847 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:56.158853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:56.159123 (+ 270us) server_negotiation.cc:407] Connection header received
0806 00:48:56.159309 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:56.159317 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:56.159361 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:56.159451 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:56.160010 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:56.160626 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:56.161508 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:56.161752 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:56.161881 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:56.162782 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:56.162786 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:56.162788 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:48:56.163256 (+ 468us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:56.163296 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:48:56.163314 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:56.163450 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:56.163562 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 00:48:56.163626 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:48:56.172540 20401 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:56.173496 20401 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:56.179546 20401 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37902f6912554cea99f84e037580ff2c: Generated new TSK 0
I20260806 00:48:56.180264 20401 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:48:56.188241 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 20276
2026-08-06T00:48:56Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1622 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:56 dist-test-slave-45dx krb5kdc[20437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:56 dist-test-slave-45dx krb5kdc[20437](info): set up 2 sockets
Aug 06 00:48:56 dist-test-slave-45dx krb5kdc[20437](info): commencing operation
krb5kdc: starting...
W20260806 00:48:57.405229 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:48:57 dist-test-slave-45dx krb5kdc[20437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:48:57.535676 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:39187
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:40519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:39187
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:57.648394 20458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:57.648656 20458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:57.648695 20458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:57.652281 20458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:48:57.652396 20458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:57.652418 20458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:57.652433 20458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:48:57.652446 20458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:48:57.657529 20458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:39187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:39187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:57.658458 20458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:57.659425 20458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:57.667912 20467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:57.667935 20466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:57.667934 20469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:57.668030 20458 server_base.cc:1034] running on GCE node
I20260806 00:48:57.668782 20458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:57.669772 20458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:57.670933 20458 hybrid_clock.cc:648] HybridClock initialized: now 1785977337670905 us; error 43 us; skew 500 ppm
Aug 06 00:48:57 dist-test-slave-45dx krb5kdc[20437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977337, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:57.673727 20458 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:48:57.674521 20458 webserver.cc:460] Webserver started at http://127.15.147.190:36631/ using document root <none> and password file <none>
I20260806 00:48:57.675015 20458 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:57.675084 20458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:57.675240 20458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:57.677232 20458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "af4d616127824cc3bc41992f55b008d0"
format_stamp: "Formatted at 2026-08-06 00:48:57 on dist-test-slave-45dx"
server_key: "2597e045564a34fbac3bfe6643481752"
server_key_iv: "355a8bb998e7f6b3a6e18f88d0284aa2"
server_key_version: "encryptionkey@0"
I20260806 00:48:57.677758 20458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "af4d616127824cc3bc41992f55b008d0"
format_stamp: "Formatted at 2026-08-06 00:48:57 on dist-test-slave-45dx"
server_key: "2597e045564a34fbac3bfe6643481752"
server_key_iv: "355a8bb998e7f6b3a6e18f88d0284aa2"
server_key_version: "encryptionkey@0"
I20260806 00:48:57.681131 20458 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:48:57.683573 20484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:57.684876 20458 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:48:57.685011 20458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "af4d616127824cc3bc41992f55b008d0"
format_stamp: "Formatted at 2026-08-06 00:48:57 on dist-test-slave-45dx"
server_key: "2597e045564a34fbac3bfe6643481752"
server_key_iv: "355a8bb998e7f6b3a6e18f88d0284aa2"
server_key_version: "encryptionkey@0"
I20260806 00:48:57.685125 20458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:57.710798 20458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:57.713995 20458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:57.714214 20458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:57.721585 20458 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:39187
I20260806 00:48:57.721938 20558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:39187 every 8 connection(s)
I20260806 00:48:57.722715 20458 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:48:57.725844 20559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:48:57.731498 20559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0: Bootstrap starting.
I20260806 00:48:57.732791 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 20458
I20260806 00:48:57.732939 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:48:57.733244 15950 external_mini_cluster.cc:1428] Setting key 0fbdca6f7c601ed18611d44c69623d78
I20260806 00:48:57.734597 20559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0: Neither blocks nor log segments found. Creating new log.
I20260806 00:48:57.735414 20559 log.cc:824] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0: Log is configured to *not* fsync() on all Append() calls
I20260806 00:48:57.737371 20559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0: No bootstrap required, opened a new log
I20260806 00:48:57.740945 20559 raft_consensus.cc:348] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af4d616127824cc3bc41992f55b008d0" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39187 } }
I20260806 00:48:57.741156 20559 raft_consensus.cc:374] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:48:57.741218 20559 raft_consensus.cc:729] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af4d616127824cc3bc41992f55b008d0, State: Initialized, Role: FOLLOWER
I20260806 00:48:57.741720 20559 consensus_queue.cc:260] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af4d616127824cc3bc41992f55b008d0" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39187 } }
Aug 06 00:48:57 dist-test-slave-45dx krb5kdc[20437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:57.741904 20559 raft_consensus.cc:388] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:48:57.741994 20559 raft_consensus.cc:482] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:48:57.742108 20559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:48:57.743249 20559 raft_consensus.cc:504] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af4d616127824cc3bc41992f55b008d0" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39187 } }
I20260806 00:48:57.743649 20559 leader_election.cc:302] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af4d616127824cc3bc41992f55b008d0; no voters:
I20260806 00:48:57.744051 20559 leader_election.cc:288] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:48:57.744166 20572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:48:57.744395 20572 raft_consensus.cc:686] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0 [term 1 LEADER]: Becoming Leader. State: Replica: af4d616127824cc3bc41992f55b008d0, State: Running, Role: LEADER
I20260806 00:48:57.744786 20572 consensus_queue.cc:237] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af4d616127824cc3bc41992f55b008d0" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39187 } }
I20260806 00:48:57.745440 20559 sys_catalog.cc:564] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:48:57.746686 20574 sys_catalog.cc:455] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af4d616127824cc3bc41992f55b008d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af4d616127824cc3bc41992f55b008d0" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39187 } } }
I20260806 00:48:57.746827 20574 sys_catalog.cc:458] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:57.747118 20576 sys_catalog.cc:455] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af4d616127824cc3bc41992f55b008d0. Latest consensus state: current_term: 1 leader_uuid: "af4d616127824cc3bc41992f55b008d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af4d616127824cc3bc41992f55b008d0" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39187 } } }
I20260806 00:48:57.747210 20576 sys_catalog.cc:458] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0 [sys.catalog]: This master's current role is: LEADER
I20260806 00:48:57.748405 20586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:48:57.748778 20568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:57.734504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36786 (local address 127.15.147.190:39187)
0806 00:48:57.734965 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:57.734981 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:57.735018 (+ 37us) server_negotiation.cc:407] Connection header received
0806 00:48:57.735734 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:57.735753 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:57.736058 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:57.736418 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:57.737215 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:57.738199 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:57.739113 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:57.739614 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:57.739781 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:57.742621 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:57.742626 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:57.742639 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:57.742665 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:57.744808 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:57.745370 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:57.745376 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:57.745382 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:57.745528 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:57.745836 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:57.745839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:57.745842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:57.746417 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:57.746639 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:57.747195 (+ 556us) server_negotiation.cc:299] Negotiation successful
0806 00:48:57.747510 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":135,"threads_started":1}
I20260806 00:48:57.751371 20586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:48:57.756286 20586 catalog_manager.cc:1347] Generated new cluster ID: 3d47fb8488cb4dd7a97339dcee033bd5
I20260806 00:48:57.756371 20586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:48:57.780968 20586 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:48:57.782056 20586 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:48:57.792407 20586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af4d616127824cc3bc41992f55b008d0: Generated new TSK 0
I20260806 00:48:57.793159 20586 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
I20260806 00:48:57.831568 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39187
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:40519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:57.942431 20603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:57.942685 20603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:57.942728 20603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:57.946979 20603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:57.947052 20603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:57.947149 20603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:48:57.953285 20603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:57.954835 20603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:57.955919 20603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:57.965639 20614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:57.965687 20617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:57.965965 20603 server_base.cc:1034] running on GCE node
W20260806 00:48:57.966140 20619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:57.966511 20603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:57.967088 20603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:57.968279 20603 hybrid_clock.cc:648] HybridClock initialized: now 1785977337968264 us; error 33 us; skew 500 ppm
Aug 06 00:48:57 dist-test-slave-45dx krb5kdc[20437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977337, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:57.971113 20603 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:48:57.971896 20603 webserver.cc:460] Webserver started at http://127.15.147.129:39477/ using document root <none> and password file <none>
I20260806 00:48:57.972479 20603 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:57.972572 20603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:57.972806 20603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:57.974721 20603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "76db062bbd044be2963a940727f6f28a"
format_stamp: "Formatted at 2026-08-06 00:48:57 on dist-test-slave-45dx"
server_key: "1943e94897b6f68690cb2ce9d8643e3c"
server_key_iv: "b68bff816fa216584b4fc4a0912465f6"
server_key_version: "encryptionkey@0"
I20260806 00:48:57.975308 20603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "76db062bbd044be2963a940727f6f28a"
format_stamp: "Formatted at 2026-08-06 00:48:57 on dist-test-slave-45dx"
server_key: "1943e94897b6f68690cb2ce9d8643e3c"
server_key_iv: "b68bff816fa216584b4fc4a0912465f6"
server_key_version: "encryptionkey@0"
I20260806 00:48:57.979014 20603 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 00:48:57.981274 20630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:57.982436 20603 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:48:57.982573 20603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "76db062bbd044be2963a940727f6f28a"
format_stamp: "Formatted at 2026-08-06 00:48:57 on dist-test-slave-45dx"
server_key: "1943e94897b6f68690cb2ce9d8643e3c"
server_key_iv: "b68bff816fa216584b4fc4a0912465f6"
server_key_version: "encryptionkey@0"
I20260806 00:48:57.982681 20603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:57.999276 20603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:58.002529 20603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:58.002730 20603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:58.003309 20603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:48:58.004114 20603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:48:58.004186 20603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:58.004256 20603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:48:58.004300 20603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:58.013108 20603 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:37441
I20260806 00:48:58.013132 20803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:37441 every 8 connection(s)
I20260806 00:48:58.013955 20603 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:48:58.018471 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 20603
I20260806 00:48:58.018601 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:48:58.018922 15950 external_mini_cluster.cc:1428] Setting key 3369c362bd9cdcacbae106c3f24e1416
Aug 06 00:48:58 dist-test-slave-45dx krb5kdc[20437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977337, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:58.029104 20568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:58.015661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:50601 (local address 127.15.147.190:39187)
0806 00:48:58.015824 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:58.015829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:58.016709 (+ 880us) server_negotiation.cc:407] Connection header received
0806 00:48:58.018092 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:58.018095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:58.018141 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:58.018214 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:58.019861 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:58.020615 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:58.022340 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:58.022600 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:58.022711 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:58.025300 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:58.025304 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:58.025306 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:58.025332 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:58.026958 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:58.027455 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:58.027459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:58.027461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:58.027550 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:58.027898 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:58.027901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:58.027903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:58.028086 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:58.028193 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:58.028781 (+ 588us) server_negotiation.cc:299] Negotiation successful
0806 00:48:58.028915 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
WARNING: no policy specified for kudu/127.15.147.130@KRBTEST.COM; defaulting to no policy
I20260806 00:48:58.029878 20814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:58.016021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:39187 (local address 127.15.147.129:50601)
0806 00:48:58.016570 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:58.016610 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:58.017902 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:58.018456 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:58.018466 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:58.018973 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:58.019600 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:58.019624 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:58.020746 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:58.020752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:58.022209 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:58.022225 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:58.022762 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:58.022767 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:58.022950 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:58.023560 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:58.023576 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:58.025140 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:58.027078 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:58.027084 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:58.027097 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:58.027351 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:58.027642 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:58.027645 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:58.027647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:58.027789 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:58.028193 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:58.028200 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:58.028543 (+ 343us) client_negotiation.cc:769] Sending connection context
0806 00:48:58.028722 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 00:48:58.029036 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":142,"threads_started":1}
I20260806 00:48:58.031203 20804 heartbeater.cc:347] Connected to a master server at 127.15.147.190:39187
I20260806 00:48:58.031471 20804 heartbeater.cc:464] Registering TS with master...
I20260806 00:48:58.032027 20804 heartbeater.cc:511] Master 127.15.147.190:39187 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.130@KRBTEST.COM" created.
I20260806 00:48:58.033780 20509 ts_manager.cc:194] Registered new tserver with Master: 76db062bbd044be2963a940727f6f28a (127.15.147.129:37441)
I20260806 00:48:58.035087 20509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:50601
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
I20260806 00:48:58.049610 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39187
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:40519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:58.163565 20818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:58.163801 20818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:58.163899 20818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:58.167896 20818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:58.167949 20818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:58.168037 20818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:48:58.173390 20818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:58.174350 20818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:58.175369 20818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:58.184813 20824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:58.184895 20832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:58.184803 20827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:58.185110 20818 server_base.cc:1034] running on GCE node
I20260806 00:48:58.185712 20818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:58.186342 20818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:58.187546 20818 hybrid_clock.cc:648] HybridClock initialized: now 1785977338187508 us; error 47 us; skew 500 ppm
Aug 06 00:48:58 dist-test-slave-45dx krb5kdc[20437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977338, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:58.190487 20818 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:48:58.191402 20818 webserver.cc:460] Webserver started at http://127.15.147.130:33615/ using document root <none> and password file <none>
I20260806 00:48:58.191964 20818 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:58.192027 20818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:58.192240 20818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:58.194299 20818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "c0abfeac16c44c5c98dab82f2419c97c"
format_stamp: "Formatted at 2026-08-06 00:48:58 on dist-test-slave-45dx"
server_key: "99c21e34fcca2f01d7ff8944be432f19"
server_key_iv: "c4c9eaa3001718c9ec707a1b58507e72"
server_key_version: "encryptionkey@0"
I20260806 00:48:58.194907 20818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "c0abfeac16c44c5c98dab82f2419c97c"
format_stamp: "Formatted at 2026-08-06 00:48:58 on dist-test-slave-45dx"
server_key: "99c21e34fcca2f01d7ff8944be432f19"
server_key_iv: "c4c9eaa3001718c9ec707a1b58507e72"
server_key_version: "encryptionkey@0"
I20260806 00:48:58.198858 20818 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 00:48:58.201368 20841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:58.202592 20818 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:48:58.202720 20818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "c0abfeac16c44c5c98dab82f2419c97c"
format_stamp: "Formatted at 2026-08-06 00:48:58 on dist-test-slave-45dx"
server_key: "99c21e34fcca2f01d7ff8944be432f19"
server_key_iv: "c4c9eaa3001718c9ec707a1b58507e72"
server_key_version: "encryptionkey@0"
I20260806 00:48:58.202832 20818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:58.219471 20818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:58.222146 20818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:58.222334 20818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:58.222929 20818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:48:58.223752 20818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:48:58.223800 20818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:58.223873 20818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:48:58.223923 20818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:58.233065 20818 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:45113
I20260806 00:48:58.233098 21014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:45113 every 8 connection(s)
I20260806 00:48:58.233968 20818 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:48:58.236582 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 20818
I20260806 00:48:58.236725 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:48:58.236999 15950 external_mini_cluster.cc:1428] Setting key b3e8341ed6e0052bfdd5a36e94690533
Aug 06 00:48:58 dist-test-slave-45dx krb5kdc[20437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977338, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.147.131@KRBTEST.COM; defaulting to no policy
I20260806 00:48:58.249317 20568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:58.235761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:49137 (local address 127.15.147.190:39187)
0806 00:48:58.235899 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:58.235903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:58.236728 (+ 825us) server_negotiation.cc:407] Connection header received
0806 00:48:58.238387 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:58.238391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:58.238455 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:58.238548 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:58.240100 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:58.240704 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:58.241889 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:58.242235 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:58.242320 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:58.245214 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:58.245217 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:58.245219 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:58.245243 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:58.246940 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:58.247613 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:58.247617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:58.247620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:58.247723 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:58.248146 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:58.248148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:58.248150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:58.248373 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:58.248458 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:58.249057 (+ 599us) server_negotiation.cc:299] Negotiation successful
0806 00:48:58.249196 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:48:58.250190 21018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:58.236064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:39187 (local address 127.15.147.130:49137)
0806 00:48:58.236571 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:58.236625 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:58.238147 (+ 1522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:58.238737 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:58.238745 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:58.239259 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:58.239951 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:58.239972 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:58.240864 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:58.240870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:58.241727 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:58.241737 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:58.242370 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:58.242373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:58.242514 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:58.243141 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:58.243157 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:58.245033 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:58.247054 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:58.247061 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:58.247078 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:58.247498 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:58.247839 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:58.247846 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:58.247850 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:58.248032 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:58.248491 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:58.248500 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:58.248810 (+ 310us) client_negotiation.cc:769] Sending connection context
0806 00:48:58.249020 (+ 210us) client_negotiation.cc:240] Negotiation successful
0806 00:48:58.249364 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":139,"threads_started":1}
I20260806 00:48:58.251322 21016 heartbeater.cc:347] Connected to a master server at 127.15.147.190:39187
I20260806 00:48:58.251609 21016 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.131@KRBTEST.COM" created.
I20260806 00:48:58.252166 21016 heartbeater.cc:511] Master 127.15.147.190:39187 requested a full tablet report, sending...
I20260806 00:48:58.253320 20509 ts_manager.cc:194] Registered new tserver with Master: c0abfeac16c44c5c98dab82f2419c97c (127.15.147.130:45113)
I20260806 00:48:58.254017 20509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:49137
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
I20260806 00:48:58.269766 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39187
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:40519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:48:58.386013 21022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:48:58.386245 21022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:48:58.386313 21022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:48:58.390358 21022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:48:58.390412 21022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:48:58.390511 21022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:48:58.395637 21022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:48:58.396680 21022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:48:58.397655 21022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:48:58.407588 21031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:48:58.407598 21032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:58.407716 21022 server_base.cc:1034] running on GCE node
W20260806 00:48:58.407588 21036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:48:58.408231 21022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:48:58.408943 21022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:48:58.410169 21022 hybrid_clock.cc:648] HybridClock initialized: now 1785977338410148 us; error 30 us; skew 500 ppm
Aug 06 00:48:58 dist-test-slave-45dx krb5kdc[20437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977338, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:48:58.413085 21022 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:48:58.414001 21022 webserver.cc:460] Webserver started at http://127.15.147.131:46757/ using document root <none> and password file <none>
I20260806 00:48:58.414544 21022 fs_manager.cc:359] Metadata directory not provided
I20260806 00:48:58.414606 21022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:48:58.414852 21022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:48:58.416997 21022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "a550bc8e3fd949cda3bb34cb9fa2472d"
format_stamp: "Formatted at 2026-08-06 00:48:58 on dist-test-slave-45dx"
server_key: "be19c0857589b2992625c0c39d813c18"
server_key_iv: "fa3c41efbf4ac9f6af5a075b311cc325"
server_key_version: "encryptionkey@0"
I20260806 00:48:58.417562 21022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "a550bc8e3fd949cda3bb34cb9fa2472d"
format_stamp: "Formatted at 2026-08-06 00:48:58 on dist-test-slave-45dx"
server_key: "be19c0857589b2992625c0c39d813c18"
server_key_iv: "fa3c41efbf4ac9f6af5a075b311cc325"
server_key_version: "encryptionkey@0"
I20260806 00:48:58.421257 21022 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:48:58.423622 21044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:58.424981 21022 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:48:58.425133 21022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "a550bc8e3fd949cda3bb34cb9fa2472d"
format_stamp: "Formatted at 2026-08-06 00:48:58 on dist-test-slave-45dx"
server_key: "be19c0857589b2992625c0c39d813c18"
server_key_iv: "fa3c41efbf4ac9f6af5a075b311cc325"
server_key_version: "encryptionkey@0"
I20260806 00:48:58.425232 21022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:48:58.452448 21022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:48:58.456589 21022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:48:58.456859 21022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:48:58.457576 21022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:48:58.458515 21022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:48:58.458592 21022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:58.458667 21022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:48:58.458710 21022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:48:58.468614 21022 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:35519
I20260806 00:48:58.468649 21213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:35519 every 8 connection(s)
I20260806 00:48:58.469486 21022 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:48:58.477888 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 21022
I20260806 00:48:58.477982 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:48:58.478231 15950 external_mini_cluster.cc:1428] Setting key 9433eaaf5fa398b30c0feae9b7ab1632
Aug 06 00:48:58 dist-test-slave-45dx krb5kdc[20437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977338, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:58.482885 20568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:58.471249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:34963 (local address 127.15.147.190:39187)
0806 00:48:58.471386 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:58.471389 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:58.472234 (+ 845us) server_negotiation.cc:407] Connection header received
0806 00:48:58.473693 (+ 1459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:58.473696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:58.473743 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:58.473824 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:58.474941 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:58.475458 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:58.476652 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:58.476866 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:58.476942 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:58.479502 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:58.479506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:58.479508 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:58.479536 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:58.481111 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:58.481569 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:58.481572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:58.481573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:58.481649 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:58.482008 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:58.482010 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:58.482012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:58.482179 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:58.482253 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:58.482679 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 00:48:58.482768 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:48:58.483693 21217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:58.471619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:39187 (local address 127.15.147.131:34963)
0806 00:48:58.472080 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0806 00:48:58.472123 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 00:48:58.473460 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:48:58.473985 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:48:58.473990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:48:58.474334 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:48:58.474816 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:58.474829 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:58.475602 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:58.475604 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:58.476539 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:48:58.476563 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:48:58.477002 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:48:58.477005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:48:58.477132 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:58.477700 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:48:58.477714 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:48:58.479337 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:48:58.481214 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:58.481218 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:58.481228 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:58.481483 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:58.481727 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:48:58.481729 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:48:58.481731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:48:58.481914 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:48:58.482265 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:48:58.482269 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:48:58.482483 (+ 214us) client_negotiation.cc:769] Sending connection context
0806 00:48:58.482629 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 00:48:58.482878 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":108,"threads_started":1}
I20260806 00:48:58.484824 21214 heartbeater.cc:347] Connected to a master server at 127.15.147.190:39187
I20260806 00:48:58.485055 21214 heartbeater.cc:464] Registering TS with master...
I20260806 00:48:58.485536 21214 heartbeater.cc:511] Master 127.15.147.190:39187 requested a full tablet report, sending...
I20260806 00:48:58.486560 20509 ts_manager.cc:194] Registered new tserver with Master: a550bc8e3fd949cda3bb34cb9fa2472d (127.15.147.131:35519)
I20260806 00:48:58.487125 20509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:34963
I20260806 00:48:58.492139 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 00:48:58 dist-test-slave-45dx krb5kdc[20437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977338, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 00:48:58 dist-test-slave-45dx krb5kdc[20437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977338, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:48:58.518949 20568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:48:58.510662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36788 (local address 127.15.147.190:39187)
0806 00:48:58.510794 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:48:58.510797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:48:58.510931 (+ 134us) server_negotiation.cc:407] Connection header received
0806 00:48:58.511044 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:48:58.511047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:48:58.511091 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:48:58.511167 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:48:58.511846 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:58.512436 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:58.513209 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:48:58.513423 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:48:58.513497 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:48:58.515924 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:48:58.515926 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:48:58.515928 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:48:58.515950 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:48:58.517261 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:58.517746 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:58.517750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:58.517751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:58.517859 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:48:58.518134 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:48:58.518136 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:48:58.518137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:48:58.518391 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:48:58.518500 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 00:48:58.518690 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 00:48:58.518787 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 00:48:58.519412 20509 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36788
I20260806 00:48:58.520254 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 20603
I20260806 00:48:58.533629 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 20818
I20260806 00:48:58.543877 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 21022
I20260806 00:48:58.555027 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 20458
2026-08-06T00:48:58Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2367 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:48:58 dist-test-slave-45dx krb5kdc[21227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:48:58 dist-test-slave-45dx krb5kdc[21227](info): set up 2 sockets
Aug 06 00:48:58 dist-test-slave-45dx krb5kdc[21227](info): commencing operation
krb5kdc: starting...
W20260806 00:48:59.771903 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:48:59 dist-test-slave-45dx krb5kdc[21227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:48:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:48:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:48:59.898972 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:45405
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:36733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:45405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:00.013028 21247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:00.013339 21247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:00.013439 21247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:00.017518 21247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:49:00.017603 21247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:00.017643 21247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:00.017702 21247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:49:00.017743 21247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:49:00.023108 21247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:36733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:45405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:45405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:00.024194 21247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:00.025255 21247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:00.034051 21253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:00.034031 21254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:00.034245 21256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:00.034228 21247 server_base.cc:1034] running on GCE node
I20260806 00:49:00.035032 21247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:00.036100 21247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:00.037276 21247 hybrid_clock.cc:648] HybridClock initialized: now 1785977340037250 us; error 37 us; skew 500 ppm
Aug 06 00:49:00 dist-test-slave-45dx krb5kdc[21227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977340, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:00.045176 21247 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:49:00.046180 21247 webserver.cc:460] Webserver started at http://127.15.147.190:43515/ using document root <none> and password file <none>
I20260806 00:49:00.046804 21247 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:00.046904 21247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:00.047153 21247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:00.049350 21247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "b106213a1a1c4ec0a7cd30bd9321bc81"
format_stamp: "Formatted at 2026-08-06 00:49:00 on dist-test-slave-45dx"
server_key: "85047cf32524f3f49a427313549a388a"
server_key_iv: "3c374d938fd5da4c095c729d46835971"
server_key_version: "encryptionkey@0"
I20260806 00:49:00.049945 21247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "b106213a1a1c4ec0a7cd30bd9321bc81"
format_stamp: "Formatted at 2026-08-06 00:49:00 on dist-test-slave-45dx"
server_key: "85047cf32524f3f49a427313549a388a"
server_key_iv: "3c374d938fd5da4c095c729d46835971"
server_key_version: "encryptionkey@0"
I20260806 00:49:00.053766 21247 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:49:00.056224 21265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:00.057444 21247 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:49:00.057591 21247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "b106213a1a1c4ec0a7cd30bd9321bc81"
format_stamp: "Formatted at 2026-08-06 00:49:00 on dist-test-slave-45dx"
server_key: "85047cf32524f3f49a427313549a388a"
server_key_iv: "3c374d938fd5da4c095c729d46835971"
server_key_version: "encryptionkey@0"
I20260806 00:49:00.057708 21247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:00.075304 21247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:00.078547 21247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:00.078784 21247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:00.086736 21247 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:45405
I20260806 00:49:00.086767 21339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:45405 every 8 connection(s)
I20260806 00:49:00.087761 21247 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:49:00.091053 21342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:00.095399 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 21247
I20260806 00:49:00.095505 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:49:00.095793 15950 external_mini_cluster.cc:1428] Setting key af2e56d90f0ed9deb06859397eb012a0
I20260806 00:49:00.096820 21342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81: Bootstrap starting.
I20260806 00:49:00.099195 21342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:00.099982 21342 log.cc:824] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:00.101976 21342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81: No bootstrap required, opened a new log
Aug 06 00:49:00 dist-test-slave-45dx krb5kdc[21227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:49:00.106598 21342 raft_consensus.cc:348] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b106213a1a1c4ec0a7cd30bd9321bc81" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 45405 } }
I20260806 00:49:00.106881 21342 raft_consensus.cc:374] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:00.106952 21342 raft_consensus.cc:729] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b106213a1a1c4ec0a7cd30bd9321bc81, State: Initialized, Role: FOLLOWER
I20260806 00:49:00.107411 21342 consensus_queue.cc:260] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b106213a1a1c4ec0a7cd30bd9321bc81" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 45405 } }
I20260806 00:49:00.107571 21342 raft_consensus.cc:388] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:49:00.107666 21342 raft_consensus.cc:482] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:49:00.107785 21342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:00.109035 21342 raft_consensus.cc:504] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b106213a1a1c4ec0a7cd30bd9321bc81" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 45405 } }
I20260806 00:49:00.109378 21342 leader_election.cc:302] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b106213a1a1c4ec0a7cd30bd9321bc81; no voters:
I20260806 00:49:00.109786 21342 leader_election.cc:288] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:49:00.109897 21347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:00.110180 21347 raft_consensus.cc:686] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81 [term 1 LEADER]: Becoming Leader. State: Replica: b106213a1a1c4ec0a7cd30bd9321bc81, State: Running, Role: LEADER
I20260806 00:49:00.110546 21347 consensus_queue.cc:237] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b106213a1a1c4ec0a7cd30bd9321bc81" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 45405 } }
I20260806 00:49:00.111110 21342 sys_catalog.cc:564] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:49:00.111423 21345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:00.097305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51178 (local address 127.15.147.190:45405)
0806 00:49:00.097782 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:00.097798 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:00.097836 (+ 38us) server_negotiation.cc:407] Connection header received
0806 00:49:00.098467 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:00.098483 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:00.098799 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:00.099141 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:00.099988 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:00.101312 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:00.102207 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:00.102883 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:00.103062 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:00.105932 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:00.105937 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:00.105950 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:00.105977 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:00.107949 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:00.108485 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:00.108489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:00.108493 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:00.108606 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:00.108936 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:00.108938 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:00.108939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:00.109391 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:00.109617 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:00.109954 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 00:49:00.110454 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":158,"threads_started":1}
I20260806 00:49:00.112258 21349 sys_catalog.cc:455] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b106213a1a1c4ec0a7cd30bd9321bc81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b106213a1a1c4ec0a7cd30bd9321bc81" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 45405 } } }
I20260806 00:49:00.112409 21349 sys_catalog.cc:458] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:00.112649 21349 sys_catalog.cc:455] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b106213a1a1c4ec0a7cd30bd9321bc81. Latest consensus state: current_term: 1 leader_uuid: "b106213a1a1c4ec0a7cd30bd9321bc81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b106213a1a1c4ec0a7cd30bd9321bc81" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 45405 } } }
I20260806 00:49:00.112725 21349 sys_catalog.cc:458] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:00.113940 21362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:49:00.116706 21362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:49:00.121882 21362 catalog_manager.cc:1347] Generated new cluster ID: 0e0795498d114a099f1871a617a6bd09
I20260806 00:49:00.121958 21362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:49:00.131488 21362 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:49:00.132416 21362 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:49:00.137287 21362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b106213a1a1c4ec0a7cd30bd9321bc81: Generated new TSK 0
I20260806 00:49:00.137945 21362 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
I20260806 00:49:00.171765 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:45405
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:36733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:00.289155 21379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:00.289407 21379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:00.289477 21379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:00.293831 21379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:00.293892 21379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:00.293998 21379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:49:00.299314 21379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:36733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:45405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:00.300416 21379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:00.301477 21379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:00.311401 21389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:00.311398 21388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:00.311447 21379 server_base.cc:1034] running on GCE node
W20260806 00:49:00.311630 21391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:00.312072 21379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:00.312856 21379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:00.314050 21379 hybrid_clock.cc:648] HybridClock initialized: now 1785977340314019 us; error 41 us; skew 500 ppm
Aug 06 00:49:00 dist-test-slave-45dx krb5kdc[21227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977340, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:00.317001 21379 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:49:00.317821 21379 webserver.cc:460] Webserver started at http://127.15.147.129:42169/ using document root <none> and password file <none>
I20260806 00:49:00.318370 21379 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:00.318463 21379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:00.318688 21379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:00.320690 21379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "0b172c0015354eb4b66e6d6138910b7e"
format_stamp: "Formatted at 2026-08-06 00:49:00 on dist-test-slave-45dx"
server_key: "154a763026ae4b968187e5e4c59d100d"
server_key_iv: "4079bd265776adce9316c26c5033cc1e"
server_key_version: "encryptionkey@0"
I20260806 00:49:00.321254 21379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "0b172c0015354eb4b66e6d6138910b7e"
format_stamp: "Formatted at 2026-08-06 00:49:00 on dist-test-slave-45dx"
server_key: "154a763026ae4b968187e5e4c59d100d"
server_key_iv: "4079bd265776adce9316c26c5033cc1e"
server_key_version: "encryptionkey@0"
I20260806 00:49:00.324848 21379 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:49:00.327195 21403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:00.328504 21379 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:49:00.328634 21379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "0b172c0015354eb4b66e6d6138910b7e"
format_stamp: "Formatted at 2026-08-06 00:49:00 on dist-test-slave-45dx"
server_key: "154a763026ae4b968187e5e4c59d100d"
server_key_iv: "4079bd265776adce9316c26c5033cc1e"
server_key_version: "encryptionkey@0"
I20260806 00:49:00.328747 21379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:00.344513 21379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:00.347981 21379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:00.348203 21379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:00.348860 21379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:00.349711 21379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:00.349782 21379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:00.349848 21379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:00.349891 21379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:00.359153 21379 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:35445
I20260806 00:49:00.359222 21568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:35445 every 8 connection(s)
I20260806 00:49:00.360150 21379 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:49:00.369535 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 21379
Aug 06 00:49:00 dist-test-slave-45dx krb5kdc[21227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977340, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:49:00.369671 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:49:00.370002 15950 external_mini_cluster.cc:1428] Setting key 3f605c1a0c8461bcabadcfceefb73a27
I20260806 00:49:00.374315 21345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:00.362080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:52537 (local address 127.15.147.190:45405)
0806 00:49:00.362240 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:00.362244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:00.363101 (+ 857us) server_negotiation.cc:407] Connection header received
0806 00:49:00.364513 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:00.364516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:00.364565 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:00.364642 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:00.365838 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:00.366373 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:00.367465 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:00.367677 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:00.367751 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:00.370299 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:00.370303 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:00.370306 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:00.370333 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:00.372096 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:00.372664 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:00.372671 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:00.372675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:00.372773 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:00.373099 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:00.373105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:00.373109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:00.373393 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:00.373501 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:00.374082 (+ 581us) server_negotiation.cc:299] Negotiation successful
0806 00:49:00.374184 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 00:49:00.375464 21572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:00.362486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45405 (local address 127.15.147.129:52537)
0806 00:49:00.362943 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:00.362988 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:00.364256 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:00.364798 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:00.364804 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:00.365168 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:00.365669 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:00.365682 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:00.366505 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:00.366508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:00.367363 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:00.367372 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:00.367803 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:00.367805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:00.367960 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:00.368574 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:00.368590 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:00.370116 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:00.372214 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:00.372219 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:00.372229 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:00.372566 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:00.372869 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:00.372871 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:00.372873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:00.373001 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:00.373497 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:00.373501 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:00.373797 (+ 296us) client_negotiation.cc:769] Sending connection context
0806 00:49:00.374014 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 00:49:00.374361 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":108,"threads_started":1}
I20260806 00:49:00.376765 21570 heartbeater.cc:347] Connected to a master server at 127.15.147.190:45405
I20260806 00:49:00.377014 21570 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:00.377522 21570 heartbeater.cc:511] Master 127.15.147.190:45405 requested a full tablet report, sending...
I20260806 00:49:00.379065 21286 ts_manager.cc:194] Registered new tserver with Master: 0b172c0015354eb4b66e6d6138910b7e (127.15.147.129:35445)
I20260806 00:49:00.380203 21286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:52537
WARNING: no policy specified for kudu/127.15.147.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
I20260806 00:49:00.400035 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:45405
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:36733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:00.511762 21578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:00.512027 21578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:00.512070 21578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:00.516103 21578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:00.516155 21578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:00.516217 21578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:49:00.521138 21578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:36733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:45405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:00.522066 21578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:00.522965 21578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:00.532606 21591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:00.532521 21585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:00.532658 21578 server_base.cc:1034] running on GCE node
W20260806 00:49:00.532521 21587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:00.533219 21578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:00.533825 21578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:00.535018 21578 hybrid_clock.cc:648] HybridClock initialized: now 1785977340534993 us; error 41 us; skew 500 ppm
Aug 06 00:49:00 dist-test-slave-45dx krb5kdc[21227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977340, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:00.537788 21578 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:49:00.538647 21578 webserver.cc:460] Webserver started at http://127.15.147.130:38549/ using document root <none> and password file <none>
I20260806 00:49:00.539148 21578 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:00.539204 21578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:00.539366 21578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:00.541339 21578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "ab3b7cf339324cc0a9c88a6019cb004b"
format_stamp: "Formatted at 2026-08-06 00:49:00 on dist-test-slave-45dx"
server_key: "358b54df9ce5a829c95c5d0cb04fb8df"
server_key_iv: "1578941c1718f73dc27c6966839e1df0"
server_key_version: "encryptionkey@0"
I20260806 00:49:00.541873 21578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "ab3b7cf339324cc0a9c88a6019cb004b"
format_stamp: "Formatted at 2026-08-06 00:49:00 on dist-test-slave-45dx"
server_key: "358b54df9ce5a829c95c5d0cb04fb8df"
server_key_iv: "1578941c1718f73dc27c6966839e1df0"
server_key_version: "encryptionkey@0"
I20260806 00:49:00.545388 21578 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 00:49:00.547577 21602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:00.548774 21578 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:49:00.548908 21578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "ab3b7cf339324cc0a9c88a6019cb004b"
format_stamp: "Formatted at 2026-08-06 00:49:00 on dist-test-slave-45dx"
server_key: "358b54df9ce5a829c95c5d0cb04fb8df"
server_key_iv: "1578941c1718f73dc27c6966839e1df0"
server_key_version: "encryptionkey@0"
I20260806 00:49:00.549014 21578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:00.570140 21578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:00.573022 21578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:00.573236 21578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:00.573803 21578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:00.574622 21578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:00.574694 21578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:00.574766 21578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:00.574820 21578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:00.584218 21578 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:44745
I20260806 00:49:00.584272 21768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:44745 every 8 connection(s)
I20260806 00:49:00.585119 21578 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:49:00.586589 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 21578
I20260806 00:49:00.586716 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:49:00.587072 15950 external_mini_cluster.cc:1428] Setting key 1fa17ef5b6cf8203e37677269a6592f5
Aug 06 00:49:00 dist-test-slave-45dx krb5kdc[21227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977340, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.147.131@KRBTEST.COM; defaulting to no policy
I20260806 00:49:00.600041 21345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:00.586930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:44253 (local address 127.15.147.190:45405)
0806 00:49:00.587085 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:00.587089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:00.588113 (+ 1024us) server_negotiation.cc:407] Connection header received
0806 00:49:00.590127 (+ 2014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:00.590130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:00.590174 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:00.590249 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:00.591703 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:00.592255 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:00.593473 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:00.593720 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:00.593840 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:00.596482 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:00.596485 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:00.596487 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:00.596510 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:00.597919 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:00.598485 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:00.598489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:00.598491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:00.598573 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:00.598930 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:00.598934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:00.598937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:00.599143 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:00.599289 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:00.599786 (+ 497us) server_negotiation.cc:299] Negotiation successful
0806 00:49:00.599911 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.131@KRBTEST.COM" created.
I20260806 00:49:00.601069 21773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:00.587360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45405 (local address 127.15.147.130:44253)
0806 00:49:00.587957 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:00.588013 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:00.589868 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:00.590454 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:00.590465 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:00.590997 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:00.591555 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:00.591569 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:00.592431 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:00.592435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:00.593324 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:00.593334 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:00.593886 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:00.593890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:00.594082 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:00.594744 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:00.594761 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:00.596230 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:00.598041 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:00.598047 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:00.598062 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:00.598383 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:00.598666 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:00.598670 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:00.598672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:00.598815 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:00.599319 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:00.599325 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:00.599564 (+ 239us) client_negotiation.cc:769] Sending connection context
0806 00:49:00.599731 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 00:49:00.600021 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":170,"threads_started":1}
I20260806 00:49:00.602298 21770 heartbeater.cc:347] Connected to a master server at 127.15.147.190:45405
I20260806 00:49:00.602547 21770 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:00.603068 21770 heartbeater.cc:511] Master 127.15.147.190:45405 requested a full tablet report, sending...
I20260806 00:49:00.604136 21286 ts_manager.cc:194] Registered new tserver with Master: ab3b7cf339324cc0a9c88a6019cb004b (127.15.147.130:44745)
I20260806 00:49:00.604756 21286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:44253
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
I20260806 00:49:00.617308 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:45405
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:36733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:00.728035 21776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:00.728366 21776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:00.728446 21776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:00.732888 21776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:00.732949 21776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:00.733008 21776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:49:00.738471 21776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:36733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:45405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:00.739614 21776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:00.740727 21776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:49:00.750820 21776 server_base.cc:1034] running on GCE node
W20260806 00:49:00.750778 21785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:00.750794 21783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:00.750778 21782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:00.751395 21776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:00.752113 21776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:00.753304 21776 hybrid_clock.cc:648] HybridClock initialized: now 1785977340753292 us; error 66 us; skew 500 ppm
Aug 06 00:49:00 dist-test-slave-45dx krb5kdc[21227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977340, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:00.760881 21776 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:49:00.761834 21776 webserver.cc:460] Webserver started at http://127.15.147.131:39613/ using document root <none> and password file <none>
I20260806 00:49:00.762406 21776 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:00.762502 21776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:00.762741 21776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:00.764904 21776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "44881460c4f04952aab6f79fdba323a2"
format_stamp: "Formatted at 2026-08-06 00:49:00 on dist-test-slave-45dx"
server_key: "52ba44da6b1113b2e60f3e071049c29a"
server_key_iv: "3f3b28a4b3b1d11d622c51ec028e828e"
server_key_version: "encryptionkey@0"
I20260806 00:49:00.765487 21776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "44881460c4f04952aab6f79fdba323a2"
format_stamp: "Formatted at 2026-08-06 00:49:00 on dist-test-slave-45dx"
server_key: "52ba44da6b1113b2e60f3e071049c29a"
server_key_iv: "3f3b28a4b3b1d11d622c51ec028e828e"
server_key_version: "encryptionkey@0"
I20260806 00:49:00.769088 21776 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:49:00.771445 21794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:00.772679 21776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:49:00.772822 21776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "44881460c4f04952aab6f79fdba323a2"
format_stamp: "Formatted at 2026-08-06 00:49:00 on dist-test-slave-45dx"
server_key: "52ba44da6b1113b2e60f3e071049c29a"
server_key_iv: "3f3b28a4b3b1d11d622c51ec028e828e"
server_key_version: "encryptionkey@0"
I20260806 00:49:00.772951 21776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:00.793228 21776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:00.796734 21776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:00.796947 21776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:00.797556 21776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:00.798429 21776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:00.798502 21776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:00.798578 21776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:00.798621 21776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:00.808102 21776 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:37541
I20260806 00:49:00.808135 21959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:37541 every 8 connection(s)
I20260806 00:49:00.809017 21776 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:49:00.814064 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 21776
I20260806 00:49:00.814232 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:49:00.814548 15950 external_mini_cluster.cc:1428] Setting key 78906ef0413b3998cc25142d3a63e8b0
Aug 06 00:49:00 dist-test-slave-45dx krb5kdc[21227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977340, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:49:00.823089 21345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:00.810841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:56065 (local address 127.15.147.190:45405)
0806 00:49:00.810996 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:00.811000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:00.811913 (+ 913us) server_negotiation.cc:407] Connection header received
0806 00:49:00.813377 (+ 1464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:00.813380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:00.813440 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:00.813549 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:00.815092 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:00.815706 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:00.816893 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:00.817177 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:00.817286 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:00.819621 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:00.819631 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:00.819634 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:00.819660 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:00.820934 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:00.821496 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:00.821500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:00.821502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:00.821621 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:00.821987 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:00.821992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:00.821994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:00.822178 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:00.822289 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:00.822769 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 00:49:00.822908 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:49:00.823817 21962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:00.811248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45405 (local address 127.15.147.131:56065)
0806 00:49:00.811733 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:00.811775 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:00.813172 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:00.813717 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:00.813723 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:00.814261 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:00.814945 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:00.814962 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:00.815833 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:00.815836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:00.816757 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:00.816767 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:00.817337 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:00.817341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:00.817505 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:00.818044 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:00.818060 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:00.819469 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:00.821052 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:00.821058 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:00.821071 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:00.821364 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:00.821720 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:00.821723 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:00.821725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:00.821886 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:00.822287 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:00.822294 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:00.822539 (+ 245us) client_negotiation.cc:769] Sending connection context
0806 00:49:00.822725 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 00:49:00.823044 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":103,"threads_started":1}
I20260806 00:49:00.824904 21960 heartbeater.cc:347] Connected to a master server at 127.15.147.190:45405
I20260806 00:49:00.825133 21960 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:00.825663 21960 heartbeater.cc:511] Master 127.15.147.190:45405 requested a full tablet report, sending...
I20260806 00:49:00.826637 21286 ts_manager.cc:194] Registered new tserver with Master: 44881460c4f04952aab6f79fdba323a2 (127.15.147.131:37541)
I20260806 00:49:00.827284 21286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:56065
I20260806 00:49:00.828398 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 00:49:00 dist-test-slave-45dx krb5kdc[21227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 00:49:00 dist-test-slave-45dx krb5kdc[21227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.129@KRBTEST.COM
I20260806 00:49:00.863040 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:00.850738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59660 (local address 127.15.147.129:35445)
0806 00:49:00.851035 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:00.851051 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:00.851106 (+ 55us) server_negotiation.cc:407] Connection header received
0806 00:49:00.851177 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:00.851187 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:00.851432 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:00.851603 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:00.852620 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:00.853602 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:00.854688 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:00.855325 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:00.855456 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:00.858849 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:00.858859 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:00.858873 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:00.858898 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:00.860947 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:00.861456 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:00.861462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:00.861463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:00.861556 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:00.861873 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:00.861877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:00.861879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:00.862134 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:00.862421 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:00.862540 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 00:49:00.862829 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":99,"threads_started":1}
Aug 06 00:49:00 dist-test-slave-45dx krb5kdc[21227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:49:00.877023 21345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:00.866605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51182 (local address 127.15.147.190:45405)
0806 00:49:00.866734 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:00.866739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:00.866879 (+ 140us) server_negotiation.cc:407] Connection header received
0806 00:49:00.867016 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:00.867019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:00.867078 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:00.867178 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:00.867995 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:00.868848 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:00.870133 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:00.870456 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:00.870570 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:00.873558 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:00.873562 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:00.873564 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:00.873594 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:00.875032 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:00.875451 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:00.875456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:00.875457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:00.875541 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:00.875818 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:00.875822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:00.875823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:00.876040 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:00.876182 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:00.876800 (+ 618us) server_negotiation.cc:299] Negotiation successful
0806 00:49:00.876904 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 00:49:00.877455 21286 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51182
I20260806 00:49:00.878134 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 21379
I20260806 00:49:00.900818 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 21578
I20260806 00:49:00.923214 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 21776
I20260806 00:49:00.934759 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 21247
2026-08-06T00:49:00Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2378 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:49:01 dist-test-slave-45dx krb5kdc[21977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:49:01 dist-test-slave-45dx krb5kdc[21977](info): set up 2 sockets
Aug 06 00:49:01 dist-test-slave-45dx krb5kdc[21977](info): commencing operation
krb5kdc: starting...
W20260806 00:49:02.175108 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260806 00:49:02.342072 15950 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.167s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260806 00:49:02.577134 15950 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.235s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260806 00:49:02.793663 15950 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.216s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/test-user.keytab.
W20260806 00:49:03.264823 15950 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.471s user 0.001s sys 0.002s
Aug 06 00:49:03 dist-test-slave-45dx krb5kdc[21977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:49:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:49:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
W20260806 00:49:03.780520 15950 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.147.190: real 0.490s user 0.002s sys 0.002s
I20260806 00:49:03.781082 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:38669
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:38669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:03.896505 21994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:03.896757 21994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:03.896798 21994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:03.900831 21994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:49:03.900885 21994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:03.900904 21994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:03.900921 21994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:49:03.900936 21994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:49:03.906003 21994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:38669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:38669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:03.906953 21994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:03.907994 21994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:03.916401 22001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:03.916440 22000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:03.916399 22004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:03.916600 21994 server_base.cc:1034] running on GCE node
I20260806 00:49:03.917205 21994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:03.918383 21994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:03.919574 21994 hybrid_clock.cc:648] HybridClock initialized: now 1785977343919546 us; error 47 us; skew 500 ppm
Aug 06 00:49:03 dist-test-slave-45dx krb5kdc[21977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977343, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:03.922446 21994 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:49:03.923216 21994 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:03.923316 21994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:03.923563 21994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:03.925716 21994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "3402644513ca48789dcd40d5bf397565"
format_stamp: "Formatted at 2026-08-06 00:49:03 on dist-test-slave-45dx"
server_key: "1deaf1b37317c179492d46026549184b"
server_key_iv: "36e168a8cc25ed97bd0e29cb1a62fc30"
server_key_version: "encryptionkey@0"
I20260806 00:49:03.926303 21994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "3402644513ca48789dcd40d5bf397565"
format_stamp: "Formatted at 2026-08-06 00:49:03 on dist-test-slave-45dx"
server_key: "1deaf1b37317c179492d46026549184b"
server_key_iv: "36e168a8cc25ed97bd0e29cb1a62fc30"
server_key_version: "encryptionkey@0"
I20260806 00:49:03.930243 21994 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:49:03.932799 22013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:03.934116 21994 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:49:03.934373 21994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "3402644513ca48789dcd40d5bf397565"
format_stamp: "Formatted at 2026-08-06 00:49:03 on dist-test-slave-45dx"
server_key: "1deaf1b37317c179492d46026549184b"
server_key_iv: "36e168a8cc25ed97bd0e29cb1a62fc30"
server_key_version: "encryptionkey@0"
I20260806 00:49:03.934502 21994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:03.957717 21994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:03.960829 21994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:03.961040 21994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:03.968575 21994 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:38669
I20260806 00:49:03.968878 22090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:38669 every 8 connection(s)
I20260806 00:49:03.969314 21994 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:49:03.972064 22091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:03.978756 22091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565: Bootstrap starting.
I20260806 00:49:03.979108 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 21994
I20260806 00:49:03.979251 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:49:03.979547 15950 external_mini_cluster.cc:1428] Setting key 37c0db99593deb5363076c284f633261
I20260806 00:49:03.981630 22091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:03.982427 22091 log.cc:824] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:03.984632 22091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565: No bootstrap required, opened a new log
Aug 06 00:49:03 dist-test-slave-45dx krb5kdc[21977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:49:03.989207 22091 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3402644513ca48789dcd40d5bf397565" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 38669 } }
I20260806 00:49:03.989497 22091 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:03.989583 22091 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3402644513ca48789dcd40d5bf397565, State: Initialized, Role: FOLLOWER
I20260806 00:49:03.990116 22091 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3402644513ca48789dcd40d5bf397565" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 38669 } }
I20260806 00:49:03.990294 22091 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:49:03.990414 22091 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:49:03.990535 22091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:03.991916 22091 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3402644513ca48789dcd40d5bf397565" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 38669 } }
I20260806 00:49:03.992344 22091 leader_election.cc:302] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3402644513ca48789dcd40d5bf397565; no voters:
I20260806 00:49:03.992689 22091 leader_election.cc:288] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:49:03.992818 22097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:03.993047 22097 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565 [term 1 LEADER]: Becoming Leader. State: Replica: 3402644513ca48789dcd40d5bf397565, State: Running, Role: LEADER
I20260806 00:49:03.993434 22097 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3402644513ca48789dcd40d5bf397565" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 38669 } }
I20260806 00:49:03.994086 22091 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:49:03.995483 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:03.980941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51316 (local address 127.15.147.190:38669)
0806 00:49:03.981368 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:03.981393 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:03.981428 (+ 35us) server_negotiation.cc:407] Connection header received
0806 00:49:03.982147 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:03.982167 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:03.982496 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:03.982838 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:03.983690 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:03.984717 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:03.985933 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:03.986409 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:03.986525 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:03.989393 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:03.989400 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:03.989414 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:03.989445 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:03.991756 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:03.992264 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:03.992271 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:03.992276 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:03.992458 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:03.992767 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:03.992779 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:03.992781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:03.993325 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:03.993523 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:03.994147 (+ 624us) server_negotiation.cc:299] Negotiation successful
0806 00:49:03.994449 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":147,"threads_started":1}
I20260806 00:49:03.996497 22101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3402644513ca48789dcd40d5bf397565. Latest consensus state: current_term: 1 leader_uuid: "3402644513ca48789dcd40d5bf397565" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3402644513ca48789dcd40d5bf397565" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 38669 } } }
I20260806 00:49:03.996625 22100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3402644513ca48789dcd40d5bf397565" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3402644513ca48789dcd40d5bf397565" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 38669 } } }
I20260806 00:49:03.996793 22101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:03.996816 22100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565 [sys.catalog]: This master's current role is: LEADER
W20260806 00:49:03.999109 22117 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:49:03.999198 22117 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:49:03.999312 22119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:49:04.000229 22119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:49:04.005784 22119 catalog_manager.cc:1347] Generated new cluster ID: a10b31782c284ff69ef518b04144d4c5
I20260806 00:49:04.005852 22119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:49:04.031075 22119 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:49:04.032429 22119 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:49:04.039546 22119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565: Generated new TSK 0
I20260806 00:49:04.040215 22119 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
I20260806 00:49:04.075201 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:38669
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:04.193743 22127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:04.194006 22127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:04.194052 22127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:04.198184 22127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:04.198240 22127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:04.198297 22127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:49:04.203480 22127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:38669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:04.204473 22127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:04.205430 22127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:04.214799 22137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:04.214816 22132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:04.214799 22134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:04.215157 22127 server_base.cc:1034] running on GCE node
I20260806 00:49:04.215569 22127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:04.216233 22127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:04.217418 22127 hybrid_clock.cc:648] HybridClock initialized: now 1785977344217404 us; error 30 us; skew 500 ppm
Aug 06 00:49:04 dist-test-slave-45dx krb5kdc[21977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977344, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:04.220249 22127 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:49:04.220978 22127 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:04.221037 22127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:04.221243 22127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:04.223274 22127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "939cce34cd8d4a86b1b176de31349afc"
format_stamp: "Formatted at 2026-08-06 00:49:04 on dist-test-slave-45dx"
server_key: "72c2e5274cf23bbf46d6fcd90a74caf7"
server_key_iv: "23143bc32aab2505efd4ef7e579b2d98"
server_key_version: "encryptionkey@0"
I20260806 00:49:04.223860 22127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "939cce34cd8d4a86b1b176de31349afc"
format_stamp: "Formatted at 2026-08-06 00:49:04 on dist-test-slave-45dx"
server_key: "72c2e5274cf23bbf46d6fcd90a74caf7"
server_key_iv: "23143bc32aab2505efd4ef7e579b2d98"
server_key_version: "encryptionkey@0"
I20260806 00:49:04.227695 22127 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 00:49:04.230252 22145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:04.231474 22127 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:49:04.231570 22127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "939cce34cd8d4a86b1b176de31349afc"
format_stamp: "Formatted at 2026-08-06 00:49:04 on dist-test-slave-45dx"
server_key: "72c2e5274cf23bbf46d6fcd90a74caf7"
server_key_iv: "23143bc32aab2505efd4ef7e579b2d98"
server_key_version: "encryptionkey@0"
I20260806 00:49:04.231694 22127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:04.254439 22127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:04.257531 22127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:04.257741 22127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:04.258347 22127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:04.259179 22127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:04.259227 22127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:04.259295 22127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:04.259325 22127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:04.271576 22127 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:37167
I20260806 00:49:04.271600 22304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:37167 every 8 connection(s)
I20260806 00:49:04.272225 22127 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:49:04.273097 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 22127
I20260806 00:49:04.273227 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:49:04.273545 15950 external_mini_cluster.cc:1428] Setting key 58e8cf0d66d811956cfcd6f3205ee0dd
Aug 06 00:49:04 dist-test-slave-45dx krb5kdc[21977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977344, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.147.130@KRBTEST.COM; defaulting to no policy
I20260806 00:49:04.287693 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:04.274272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:53389 (local address 127.15.147.190:38669)
0806 00:49:04.274442 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:04.274446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:04.275888 (+ 1442us) server_negotiation.cc:407] Connection header received
0806 00:49:04.276836 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:04.276842 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:04.276909 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:04.277006 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:04.278215 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.279070 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.280269 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.280732 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.280862 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:04.283697 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:04.283701 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:04.283704 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:04.283739 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:04.285580 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:04.286111 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:04.286114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:04.286115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:04.286202 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:04.286611 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:04.286614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:04.286615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:04.286804 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:04.286892 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:04.287389 (+ 497us) server_negotiation.cc:299] Negotiation successful
0806 00:49:04.287551 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.130@KRBTEST.COM" created.
I20260806 00:49:04.288744 22309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:04.274708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:38669 (local address 127.15.147.129:53389)
0806 00:49:04.275710 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:04.275766 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:04.276633 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:04.277164 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:04.277170 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:04.277543 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:04.278052 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:04.278066 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.279212 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.279215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:04.280132 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:04.280142 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.280917 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.280920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:04.281070 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:04.281730 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:04.281748 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:04.283492 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:04.285701 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:04.285707 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:04.285720 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:04.286006 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:04.286285 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:04.286288 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:04.286289 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:04.286508 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:04.286900 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:04.286905 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:04.287159 (+ 254us) client_negotiation.cc:769] Sending connection context
0806 00:49:04.287333 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 00:49:04.287686 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":175,"threads_started":1}
I20260806 00:49:04.290110 22305 heartbeater.cc:347] Connected to a master server at 127.15.147.190:38669
I20260806 00:49:04.290392 22305 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:04.290968 22305 heartbeater.cc:511] Master 127.15.147.190:38669 requested a full tablet report, sending...
I20260806 00:49:04.292601 22042 ts_manager.cc:194] Registered new tserver with Master: 939cce34cd8d4a86b1b176de31349afc (127.15.147.129:37167)
I20260806 00:49:04.293745 22042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:53389
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
I20260806 00:49:04.307509 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:38669
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:04.424064 22313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:04.424365 22313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:04.424449 22313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:04.428431 22313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:04.428480 22313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:04.428570 22313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:49:04.433727 22313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:38669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:04.434721 22313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:04.435814 22313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:04.445266 22321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:04.445268 22324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:04.445251 22320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:04.445542 22313 server_base.cc:1034] running on GCE node
I20260806 00:49:04.446012 22313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:04.446590 22313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:04.447784 22313 hybrid_clock.cc:648] HybridClock initialized: now 1785977344447763 us; error 32 us; skew 500 ppm
Aug 06 00:49:04 dist-test-slave-45dx krb5kdc[21977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977344, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:04.450619 22313 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:49:04.451545 22313 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:04.451660 22313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:04.452033 22313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:04.454188 22313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "15bd7ea4ff02453b9d9954754282ca27"
format_stamp: "Formatted at 2026-08-06 00:49:04 on dist-test-slave-45dx"
server_key: "3d7c718e2c25986c6092ae1e8e0762cb"
server_key_iv: "99b866a5d4c2cd22d5cc29e97e76aa01"
server_key_version: "encryptionkey@0"
I20260806 00:49:04.454767 22313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "15bd7ea4ff02453b9d9954754282ca27"
format_stamp: "Formatted at 2026-08-06 00:49:04 on dist-test-slave-45dx"
server_key: "3d7c718e2c25986c6092ae1e8e0762cb"
server_key_iv: "99b866a5d4c2cd22d5cc29e97e76aa01"
server_key_version: "encryptionkey@0"
I20260806 00:49:04.458422 22313 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:49:04.460820 22331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:04.462060 22313 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:49:04.462212 22313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "15bd7ea4ff02453b9d9954754282ca27"
format_stamp: "Formatted at 2026-08-06 00:49:04 on dist-test-slave-45dx"
server_key: "3d7c718e2c25986c6092ae1e8e0762cb"
server_key_iv: "99b866a5d4c2cd22d5cc29e97e76aa01"
server_key_version: "encryptionkey@0"
I20260806 00:49:04.462330 22313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:04.488848 22313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:04.492113 22313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:04.492389 22313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:04.493033 22313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:04.493925 22313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:04.494009 22313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:04.494158 22313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:04.494204 22313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:04.503726 22313 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:40805
I20260806 00:49:04.503743 22486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:40805 every 8 connection(s)
I20260806 00:49:04.504465 22313 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
Aug 06 00:49:04 dist-test-slave-45dx krb5kdc[21977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977344, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:49:04.514703 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 22313
I20260806 00:49:04.514846 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:49:04.515218 15950 external_mini_cluster.cc:1428] Setting key 17565ba4060fb2464ab88434a42d48e1
I20260806 00:49:04.518321 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:04.506304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:33499 (local address 127.15.147.190:38669)
0806 00:49:04.506486 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:04.506492 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:04.507733 (+ 1241us) server_negotiation.cc:407] Connection header received
0806 00:49:04.508680 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:04.508683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:04.508731 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:04.508804 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:04.510032 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.510552 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.511686 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.511919 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.512010 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:04.514521 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:04.514524 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:04.514526 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:04.514551 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:04.516038 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:04.516656 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:04.516660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:04.516663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:04.516762 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:04.517170 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:04.517173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:04.517174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:04.517347 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:04.517445 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:04.518057 (+ 612us) server_negotiation.cc:299] Negotiation successful
0806 00:49:04.518187 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 00:49:04.519527 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:04.506743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:38669 (local address 127.15.147.130:33499)
0806 00:49:04.507579 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:04.507622 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:04.508485 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:04.508988 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:04.508994 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:04.509363 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:04.509904 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:04.509919 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.510661 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.510664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:04.511547 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:04.511556 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.512066 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.512068 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:04.512207 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:04.512857 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:04.512880 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:04.514341 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:04.516187 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:04.516192 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:04.516203 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:04.516522 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:04.516871 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:04.516875 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:04.516877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:04.517066 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:04.517481 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:04.517489 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:04.517808 (+ 319us) client_negotiation.cc:769] Sending connection context
0806 00:49:04.518011 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 00:49:04.518428 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":118,"threads_started":1}
I20260806 00:49:04.520792 22487 heartbeater.cc:347] Connected to a master server at 127.15.147.190:38669
I20260806 00:49:04.521046 22487 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:04.521485 22487 heartbeater.cc:511] Master 127.15.147.190:38669 requested a full tablet report, sending...
I20260806 00:49:04.522590 22042 ts_manager.cc:194] Registered new tserver with Master: 15bd7ea4ff02453b9d9954754282ca27 (127.15.147.130:40805)
I20260806 00:49:04.523458 22042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:33499
WARNING: no policy specified for kudu/127.15.147.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
I20260806 00:49:04.545766 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:38669
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:40813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:04.660645 22492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:04.660898 22492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:04.660966 22492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:04.665082 22492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:04.665177 22492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:04.665289 22492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:49:04.670415 22492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:38669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:04.671355 22492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:04.672463 22492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:04.681629 22497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:04.681593 22501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:04.681664 22499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:04.682103 22492 server_base.cc:1034] running on GCE node
I20260806 00:49:04.682526 22492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:04.683199 22492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:04.684388 22492 hybrid_clock.cc:648] HybridClock initialized: now 1785977344684369 us; error 76 us; skew 500 ppm
Aug 06 00:49:04 dist-test-slave-45dx krb5kdc[21977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977344, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:04.687268 22492 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:49:04.688019 22492 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:04.688114 22492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:04.688390 22492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:04.690364 22492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "1ba8238ad9cb4cfb88262939254ce5bd"
format_stamp: "Formatted at 2026-08-06 00:49:04 on dist-test-slave-45dx"
server_key: "287fd37b6875cb614149a91af063e642"
server_key_iv: "d38904071a8f6a49affa0eba5dce237c"
server_key_version: "encryptionkey@0"
I20260806 00:49:04.690954 22492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "1ba8238ad9cb4cfb88262939254ce5bd"
format_stamp: "Formatted at 2026-08-06 00:49:04 on dist-test-slave-45dx"
server_key: "287fd37b6875cb614149a91af063e642"
server_key_iv: "d38904071a8f6a49affa0eba5dce237c"
server_key_version: "encryptionkey@0"
I20260806 00:49:04.694769 22492 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 00:49:04.697227 22509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:04.698469 22492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260806 00:49:04.698624 22492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "1ba8238ad9cb4cfb88262939254ce5bd"
format_stamp: "Formatted at 2026-08-06 00:49:04 on dist-test-slave-45dx"
server_key: "287fd37b6875cb614149a91af063e642"
server_key_iv: "d38904071a8f6a49affa0eba5dce237c"
server_key_version: "encryptionkey@0"
I20260806 00:49:04.698743 22492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:04.720358 22492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:04.723893 22492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:04.724115 22492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:04.724855 22492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:04.725785 22492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:04.725859 22492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:04.725934 22492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:04.725986 22492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:04.735776 22492 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:45535
I20260806 00:49:04.735913 22665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:45535 every 8 connection(s)
I20260806 00:49:04.736570 22492 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:49:04.743235 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 22492
I20260806 00:49:04.743376 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:49:04.743655 15950 external_mini_cluster.cc:1428] Setting key 0255f951425fe14b6b638330da49cc68
Aug 06 00:49:04 dist-test-slave-45dx krb5kdc[21977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977344, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:49:04.750584 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:04.738386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:46385 (local address 127.15.147.190:38669)
0806 00:49:04.738538 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:04.738541 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:04.739897 (+ 1356us) server_negotiation.cc:407] Connection header received
0806 00:49:04.740832 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:04.740836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:04.740893 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:04.740972 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:04.742175 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.742714 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.744037 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.744414 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.744542 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:04.747351 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:04.747354 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:04.747357 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:04.747392 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:04.748864 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:04.749305 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:04.749307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:04.749309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:04.749396 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:04.749689 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:04.749691 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:04.749693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:04.749863 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:04.749939 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:04.750362 (+ 423us) server_negotiation.cc:299] Negotiation successful
0806 00:49:04.750466 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 00:49:04.751340 22669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:04.738810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:38669 (local address 127.15.147.131:46385)
0806 00:49:04.739731 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:04.739774 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:04.740611 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:04.741150 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:04.741157 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:04.741509 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:04.742022 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:04.742036 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.742846 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.742850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:04.743879 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:04.743890 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.744596 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.744603 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:04.744804 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:04.745496 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:04.745517 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:04.747197 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:04.748967 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:04.748972 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:04.748981 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:04.749218 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:04.749475 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:04.749477 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:04.749478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:04.749612 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:04.749953 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:04.749957 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:04.750165 (+ 208us) client_negotiation.cc:769] Sending connection context
0806 00:49:04.750313 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 00:49:04.750545 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":148,"threads_started":1}
I20260806 00:49:04.752493 22666 heartbeater.cc:347] Connected to a master server at 127.15.147.190:38669
I20260806 00:49:04.752718 22666 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:04.753149 22666 heartbeater.cc:511] Master 127.15.147.190:38669 requested a full tablet report, sending...
I20260806 00:49:04.754125 22042 ts_manager.cc:194] Registered new tserver with Master: 1ba8238ad9cb4cfb88262939254ce5bd (127.15.147.131:45535)
I20260806 00:49:04.754637 22042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:46385
I20260806 00:49:04.757745 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:49:04.767019 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:04.759632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51322 (local address 127.15.147.190:38669)
0806 00:49:04.759761 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:04.759765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:04.759967 (+ 202us) server_negotiation.cc:407] Connection header received
0806 00:49:04.760102 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:04.760105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:04.760148 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:04.760230 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:04.760964 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.761453 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.762294 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.762515 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.762627 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:04.763959 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:04.763962 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:04.763965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:04.763993 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:04.765263 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:04.765740 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:04.765744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:04.765745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:04.765823 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:04.766114 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:04.766118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:04.766119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:04.766325 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:04.766470 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:04.766740 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 00:49:04.766855 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:49:04.769865 22042 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51322:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:49:04.772858 22042 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:49:04.786834 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:04.780791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:40805 (local address 127.0.0.1:59344)
0806 00:49:04.781095 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:04.781146 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:04.781282 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:04.782622 (+ 1340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:04.782628 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:04.782663 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:04.782898 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:04.782905 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.784085 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.784088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:04.785389 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:04.785400 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.786197 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.786201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:04.786331 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:04.786377 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 00:49:04.786507 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 00:49:04.786659 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":72,"spinlock_wait_cycles":6144,"thread_start_us":127,"threads_started":1}
I20260806 00:49:04.786839 22688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:04.781639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59344 (local address 127.15.147.130:40805)
0806 00:49:04.782189 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:04.782202 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:04.782230 (+ 28us) server_negotiation.cc:407] Connection header received
0806 00:49:04.782305 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:04.782309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:04.782459 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:04.782628 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:04.783028 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.783957 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.785516 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.786070 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.786207 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:04.786342 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:04.786557 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 00:49:04.786666 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":209,"threads_started":1}
I20260806 00:49:04.787176 22687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:04.781346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:45535 (local address 127.0.0.1:38610)
0806 00:49:04.781712 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:04.781744 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:04.782001 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:04.782626 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:04.782629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:04.782666 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:04.782898 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:04.782908 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.784449 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.784453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:04.785616 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:04.785627 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.786753 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.786757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:04.786887 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:04.786913 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:49:04.786965 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 00:49:04.787032 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":41,"spinlock_wait_cycles":5632,"thread_start_us":152,"threads_started":1}
I20260806 00:49:04.787835 22689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:04.781667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38610 (local address 127.15.147.131:45535)
0806 00:49:04.782252 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:04.782262 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:04.782281 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:49:04.782341 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:04.782345 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:04.782486 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:04.782646 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:04.783059 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.784200 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.785764 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.786607 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.787396 (+ 789us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:04.787524 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:04.787575 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 00:49:04.787664 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":294,"threads_started":1}
I20260806 00:49:04.788053 22684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:04.780577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:37167 (local address 127.0.0.1:51022)
0806 00:49:04.781095 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:04.781145 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:04.781319 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:04.783474 (+ 2155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:04.783477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:04.783497 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:04.783592 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:04.783598 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.785058 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.785062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:04.786173 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:04.786183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.787645 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.787648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:04.787741 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:04.787764 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:49:04.787837 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 00:49:04.787918 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":3,"thread_start_us":142,"threads_started":1}
I20260806 00:49:04.788625 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:04.781278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51022 (local address 127.15.147.129:37167)
0806 00:49:04.782990 (+ 1712us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:04.782997 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:04.783020 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:49:04.783091 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:04.783097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:04.783289 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:04.783466 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:04.783722 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.784906 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.786309 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.787164 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.787468 (+ 304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:04.787583 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:04.788419 (+ 836us) server_negotiation.cc:299] Negotiation successful
0806 00:49:04.788503 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1580,"thread_start_us":66,"threads_started":1}
I20260806 00:49:04.790462 22576 tablet_service.cc:1467] Processing CreateTablet for tablet 0be7c719291b4e69996626b4f5554257 (DEFAULT_TABLE table=test-table [id=bb079eafe8c54d47815ad4abbd51dffe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:04.790462 22394 tablet_service.cc:1467] Processing CreateTablet for tablet 0be7c719291b4e69996626b4f5554257 (DEFAULT_TABLE table=test-table [id=bb079eafe8c54d47815ad4abbd51dffe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:04.791603 22576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0be7c719291b4e69996626b4f5554257. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:04.791601 22394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0be7c719291b4e69996626b4f5554257. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:04.794026 22203 tablet_service.cc:1467] Processing CreateTablet for tablet 0be7c719291b4e69996626b4f5554257 (DEFAULT_TABLE table=test-table [id=bb079eafe8c54d47815ad4abbd51dffe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:04.795078 22203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0be7c719291b4e69996626b4f5554257. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:04.799212 22690 tablet_bootstrap.cc:492] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd: Bootstrap starting.
I20260806 00:49:04.799642 22691 tablet_bootstrap.cc:492] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27: Bootstrap starting.
I20260806 00:49:04.801622 22693 tablet_bootstrap.cc:492] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc: Bootstrap starting.
I20260806 00:49:04.802266 22690 tablet_bootstrap.cc:654] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:04.802523 22691 tablet_bootstrap.cc:654] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:04.803541 22690 log.cc:824] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:04.803665 22691 log.cc:824] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:04.803850 22693 tablet_bootstrap.cc:654] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:04.804725 22693 log.cc:824] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:04.805596 22690 tablet_bootstrap.cc:492] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd: No bootstrap required, opened a new log
I20260806 00:49:04.805814 22690 ts_tablet_manager.cc:1397] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 00:49:04.806560 22691 tablet_bootstrap.cc:492] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27: No bootstrap required, opened a new log
I20260806 00:49:04.806564 22693 tablet_bootstrap.cc:492] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc: No bootstrap required, opened a new log
I20260806 00:49:04.806737 22691 ts_tablet_manager.cc:1397] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 00:49:04.806737 22693 ts_tablet_manager.cc:1397] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:49:04.808807 22690 raft_consensus.cc:348] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939cce34cd8d4a86b1b176de31349afc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 37167 } } peers { permanent_uuid: "15bd7ea4ff02453b9d9954754282ca27" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40805 } } peers { permanent_uuid: "1ba8238ad9cb4cfb88262939254ce5bd" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45535 } }
I20260806 00:49:04.809098 22690 raft_consensus.cc:374] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:04.809170 22690 raft_consensus.cc:729] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ba8238ad9cb4cfb88262939254ce5bd, State: Initialized, Role: FOLLOWER
I20260806 00:49:04.809609 22690 consensus_queue.cc:260] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939cce34cd8d4a86b1b176de31349afc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 37167 } } peers { permanent_uuid: "15bd7ea4ff02453b9d9954754282ca27" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40805 } } peers { permanent_uuid: "1ba8238ad9cb4cfb88262939254ce5bd" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45535 } }
I20260806 00:49:04.810303 22666 heartbeater.cc:503] Master 127.15.147.190:38669 was elected leader, sending a full tablet report...
I20260806 00:49:04.810655 22690 ts_tablet_manager.cc:1428] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:49:04.810607 22691 raft_consensus.cc:348] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939cce34cd8d4a86b1b176de31349afc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 37167 } } peers { permanent_uuid: "15bd7ea4ff02453b9d9954754282ca27" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40805 } } peers { permanent_uuid: "1ba8238ad9cb4cfb88262939254ce5bd" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45535 } }
I20260806 00:49:04.810894 22691 raft_consensus.cc:374] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:04.810957 22691 raft_consensus.cc:729] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15bd7ea4ff02453b9d9954754282ca27, State: Initialized, Role: FOLLOWER
I20260806 00:49:04.811044 22693 raft_consensus.cc:348] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939cce34cd8d4a86b1b176de31349afc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 37167 } } peers { permanent_uuid: "15bd7ea4ff02453b9d9954754282ca27" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40805 } } peers { permanent_uuid: "1ba8238ad9cb4cfb88262939254ce5bd" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45535 } }
I20260806 00:49:04.811286 22693 raft_consensus.cc:374] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:04.811353 22693 raft_consensus.cc:729] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939cce34cd8d4a86b1b176de31349afc, State: Initialized, Role: FOLLOWER
I20260806 00:49:04.811415 22691 consensus_queue.cc:260] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939cce34cd8d4a86b1b176de31349afc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 37167 } } peers { permanent_uuid: "15bd7ea4ff02453b9d9954754282ca27" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40805 } } peers { permanent_uuid: "1ba8238ad9cb4cfb88262939254ce5bd" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45535 } }
I20260806 00:49:04.811735 22693 consensus_queue.cc:260] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939cce34cd8d4a86b1b176de31349afc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 37167 } } peers { permanent_uuid: "15bd7ea4ff02453b9d9954754282ca27" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40805 } } peers { permanent_uuid: "1ba8238ad9cb4cfb88262939254ce5bd" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45535 } }
I20260806 00:49:04.812333 22691 ts_tablet_manager.cc:1428] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 00:49:04.812404 22305 heartbeater.cc:503] Master 127.15.147.190:38669 was elected leader, sending a full tablet report...
I20260806 00:49:04.812655 22487 heartbeater.cc:503] Master 127.15.147.190:38669 was elected leader, sending a full tablet report...
I20260806 00:49:04.813562 22693 ts_tablet_manager.cc:1428] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 00:49:04.930002 22702 raft_consensus.cc:482] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:49:04.930289 22702 raft_consensus.cc:504] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939cce34cd8d4a86b1b176de31349afc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 37167 } } peers { permanent_uuid: "15bd7ea4ff02453b9d9954754282ca27" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40805 } } peers { permanent_uuid: "1ba8238ad9cb4cfb88262939254ce5bd" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45535 } }
I20260806 00:49:04.931579 22702 leader_election.cc:288] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15bd7ea4ff02453b9d9954754282ca27 (127.15.147.130:40805), 1ba8238ad9cb4cfb88262939254ce5bd (127.15.147.131:45535)
I20260806 00:49:04.937309 22688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:04.931944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:54899 (local address 127.15.147.130:40805)
0806 00:49:04.932114 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:04.932119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:04.932474 (+ 355us) server_negotiation.cc:407] Connection header received
0806 00:49:04.932621 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:04.932624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:04.932677 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:04.932763 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:04.933172 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.933989 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.935908 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.936545 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.936675 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:04.936728 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:04.937089 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 00:49:04.937169 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 00:49:04.937307 22689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:04.932014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:36969 (local address 127.15.147.131:45535)
0806 00:49:04.932145 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:04.932149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:04.932473 (+ 324us) server_negotiation.cc:407] Connection header received
0806 00:49:04.932621 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:04.932624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:04.932677 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:04.932763 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:04.933172 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.933975 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.935907 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.936564 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.936675 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:04.936728 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:04.937090 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 00:49:04.937169 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:49:04.937408 22703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:04.931789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:40805 (local address 127.15.147.129:54899)
0806 00:49:04.932314 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:04.932352 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:04.932508 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:04.932794 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:04.932797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:04.932823 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:04.933060 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:04.933066 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.934105 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.934109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:04.935757 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:04.935770 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.936682 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.936685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:04.936909 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:04.936982 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 00:49:04.937057 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 00:49:04.937187 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":275,"spinlock_wait_cycles":14848,"thread_start_us":211,"threads_started":1}
I20260806 00:49:04.937785 22704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:04.931907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:45535 (local address 127.15.147.129:36969)
0806 00:49:04.932375 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:04.932388 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:04.932508 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:04.932794 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:04.932797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:04.932824 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:04.933046 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:04.933054 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.934105 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.934109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:04.935760 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:04.935770 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.936682 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.936685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:04.936911 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:04.936975 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 00:49:04.937057 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 00:49:04.937184 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":321,"spinlock_wait_cycles":33280,"thread_start_us":139,"threads_started":1}
I20260806 00:49:04.938136 22421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0be7c719291b4e69996626b4f5554257" candidate_uuid: "939cce34cd8d4a86b1b176de31349afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15bd7ea4ff02453b9d9954754282ca27" is_pre_election: true
I20260806 00:49:04.938500 22421 raft_consensus.cc:2454] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 939cce34cd8d4a86b1b176de31349afc in term 0.
I20260806 00:49:04.938555 22602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0be7c719291b4e69996626b4f5554257" candidate_uuid: "939cce34cd8d4a86b1b176de31349afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ba8238ad9cb4cfb88262939254ce5bd" is_pre_election: true
I20260806 00:49:04.938833 22602 raft_consensus.cc:2454] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 939cce34cd8d4a86b1b176de31349afc in term 0.
I20260806 00:49:04.939074 22150 leader_election.cc:302] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15bd7ea4ff02453b9d9954754282ca27, 939cce34cd8d4a86b1b176de31349afc; no voters:
I20260806 00:49:04.939462 22702 raft_consensus.cc:2781] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:49:04.939563 22702 raft_consensus.cc:482] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:49:04.939618 22702 raft_consensus.cc:3037] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:04.940723 22702 raft_consensus.cc:504] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939cce34cd8d4a86b1b176de31349afc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 37167 } } peers { permanent_uuid: "15bd7ea4ff02453b9d9954754282ca27" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40805 } } peers { permanent_uuid: "1ba8238ad9cb4cfb88262939254ce5bd" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45535 } }
I20260806 00:49:04.941059 22702 leader_election.cc:288] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [CANDIDATE]: Term 1 election: Requested vote from peers 15bd7ea4ff02453b9d9954754282ca27 (127.15.147.130:40805), 1ba8238ad9cb4cfb88262939254ce5bd (127.15.147.131:45535)
I20260806 00:49:04.941390 22421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0be7c719291b4e69996626b4f5554257" candidate_uuid: "939cce34cd8d4a86b1b176de31349afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15bd7ea4ff02453b9d9954754282ca27"
I20260806 00:49:04.941453 22602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0be7c719291b4e69996626b4f5554257" candidate_uuid: "939cce34cd8d4a86b1b176de31349afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ba8238ad9cb4cfb88262939254ce5bd"
I20260806 00:49:04.941533 22421 raft_consensus.cc:3037] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:04.941598 22602 raft_consensus.cc:3037] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:04.942703 22421 raft_consensus.cc:2454] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939cce34cd8d4a86b1b176de31349afc in term 1.
I20260806 00:49:04.942766 22602 raft_consensus.cc:2454] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 939cce34cd8d4a86b1b176de31349afc in term 1.
I20260806 00:49:04.943042 22150 leader_election.cc:302] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15bd7ea4ff02453b9d9954754282ca27, 939cce34cd8d4a86b1b176de31349afc; no voters:
I20260806 00:49:04.943234 22702 raft_consensus.cc:2781] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:04.943459 22702 raft_consensus.cc:686] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [term 1 LEADER]: Becoming Leader. State: Replica: 939cce34cd8d4a86b1b176de31349afc, State: Running, Role: LEADER
I20260806 00:49:04.943789 22702 consensus_queue.cc:237] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939cce34cd8d4a86b1b176de31349afc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 37167 } } peers { permanent_uuid: "15bd7ea4ff02453b9d9954754282ca27" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40805 } } peers { permanent_uuid: "1ba8238ad9cb4cfb88262939254ce5bd" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45535 } }
I20260806 00:49:04.947007 22041 catalog_manager.cc:5495] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc reported cstate change: term changed from 0 to 1, leader changed from <none> to 939cce34cd8d4a86b1b176de31349afc (127.15.147.129). New cstate: current_term: 1 leader_uuid: "939cce34cd8d4a86b1b176de31349afc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939cce34cd8d4a86b1b176de31349afc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 37167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15bd7ea4ff02453b9d9954754282ca27" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ba8238ad9cb4cfb88262939254ce5bd" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45535 } health_report { overall_health: UNKNOWN } } }
I20260806 00:49:04.976934 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:04.971840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51026 (local address 127.15.147.129:37167)
0806 00:49:04.971979 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:04.971986 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:04.972005 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:49:04.972096 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:04.972099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:04.972193 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:04.972333 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:49:04.972821 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.973608 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.974970 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:04.975323 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:04.975431 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:04.975924 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:49:04.976489 (+ 565us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:49:04.976615 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:04.976687 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 00:49:04.976765 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 00:49:04.987469 22667 tablet.cc:2369] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:49:04.987707 22421 raft_consensus.cc:1261] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27 [term 1 FOLLOWER]: Refusing update from remote peer 939cce34cd8d4a86b1b176de31349afc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:04.987716 22602 raft_consensus.cc:1261] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd [term 1 FOLLOWER]: Refusing update from remote peer 939cce34cd8d4a86b1b176de31349afc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:04.988452 22705 consensus_queue.cc:1035] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [LEADER]: Connected to new peer: Peer: permanent_uuid: "15bd7ea4ff02453b9d9954754282ca27" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:04.988695 22702 consensus_queue.cc:1035] T 0be7c719291b4e69996626b4f5554257 P 939cce34cd8d4a86b1b176de31349afc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ba8238ad9cb4cfb88262939254ce5bd" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:04.995539 22710 mvcc.cc:205] Tried to move back new op lower bound from 7315363205062488064 to 7315363204892889088. Current Snapshot: MvccSnapshot[applied={T|T < 7315363205062488064}]
I20260806 00:49:04.996662 22708 mvcc.cc:205] Tried to move back new op lower bound from 7315363205062488064 to 7315363204892889088. Current Snapshot: MvccSnapshot[applied={T|T < 7315363205062488064}]
I20260806 00:49:04.998325 22709 mvcc.cc:205] Tried to move back new op lower bound from 7315363205062488064 to 7315363204892889088. Current Snapshot: MvccSnapshot[applied={T|T < 7315363205062488064}]
I20260806 00:49:05.005196 22042 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:49:05.005383 22042 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:49:05.007803 22042 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3402644513ca48789dcd40d5bf397565: Sending DeleteTablet for 3 replicas of tablet 0be7c719291b4e69996626b4f5554257
I20260806 00:49:05.008590 22576 tablet_service.cc:1514] Processing DeleteTablet for tablet 0be7c719291b4e69996626b4f5554257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:05 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:38610
I20260806 00:49:05.008662 22394 tablet_service.cc:1514] Processing DeleteTablet for tablet 0be7c719291b4e69996626b4f5554257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:05 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:59344
I20260806 00:49:05.009173 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 22127
I20260806 00:49:05.009202 22203 tablet_service.cc:1514] Processing DeleteTablet for tablet 0be7c719291b4e69996626b4f5554257 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:05 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:51022
I20260806 00:49:05.013592 22723 tablet_replica.cc:330] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27: stopping tablet replica
I20260806 00:49:05.013916 22723 raft_consensus.cc:2229] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:05.014051 22722 tablet_replica.cc:330] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd: stopping tablet replica
I20260806 00:49:05.014228 22723 raft_consensus.cc:2258] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:05.014458 22722 raft_consensus.cc:2229] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:05.014775 22722 raft_consensus.cc:2258] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:05.016063 22722 ts_tablet_manager.cc:1905] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:05.016225 22723 ts_tablet_manager.cc:1905] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:05.018507 22722 ts_tablet_manager.cc:1918] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:49:05.018625 22722 log.cc:1196] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/0be7c719291b4e69996626b4f5554257
I20260806 00:49:05.018988 22722 ts_tablet_manager.cc:1939] T 0be7c719291b4e69996626b4f5554257 P 1ba8238ad9cb4cfb88262939254ce5bd: Deleting consensus metadata
I20260806 00:49:05.019726 22723 ts_tablet_manager.cc:1918] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:49:05.019832 22723 log.cc:1196] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/0be7c719291b4e69996626b4f5554257
I20260806 00:49:05.020071 22015 catalog_manager.cc:4841] TS 1ba8238ad9cb4cfb88262939254ce5bd (127.15.147.131:45535): tablet 0be7c719291b4e69996626b4f5554257 (table test-table [id=bb079eafe8c54d47815ad4abbd51dffe]) successfully deleted
I20260806 00:49:05.020172 22723 ts_tablet_manager.cc:1939] T 0be7c719291b4e69996626b4f5554257 P 15bd7ea4ff02453b9d9954754282ca27: Deleting consensus metadata
I20260806 00:49:05.021363 22021 catalog_manager.cc:4841] TS 15bd7ea4ff02453b9d9954754282ca27 (127.15.147.130:40805): tablet 0be7c719291b4e69996626b4f5554257 (table test-table [id=bb079eafe8c54d47815ad4abbd51dffe]) successfully deleted
W20260806 00:49:05.022142 22015 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.147.129:37167 (error 108)
I20260806 00:49:05.022419 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 22313
I20260806 00:49:05.025177 22684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:05.024448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:37167 (local address 127.0.0.1:51042)
0806 00:49:05.025044 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:05.025105 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:37167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517}
W20260806 00:49:05.025314 22015 catalog_manager.cc:4584] TS 939cce34cd8d4a86b1b176de31349afc (127.15.147.129:37167): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0be7c719291b4e69996626b4f5554257: Network error: Client connection negotiation failed: client connection to 127.15.147.129:37167: connect: Connection refused (error 111)
I20260806 00:49:05.031924 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 22492
I20260806 00:49:05.045257 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 21994
2026-08-06T00:49:05Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4109 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T00:49:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:49:05Z Disabled control of system clock
I20260806 00:49:05.081547 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:46717
--webserver_interface=127.15.147.190
--webserver_port=0
--builtin_ntp_servers=127.15.147.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:46717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 00:49:05.194986 22729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:05.195271 22729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:05.195309 22729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:05.199510 22729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:49:05.199572 22729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:05.199592 22729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:05.199612 22729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:49:05.199627 22729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:49:05.204487 22729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:46717
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:46717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:05.205636 22729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:05.207154 22729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:49:05.215987 22729 server_base.cc:1034] running on GCE node
W20260806 00:49:05.215948 22736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:05.215934 22738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:05.215922 22741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:05.216810 22729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:05.217898 22729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:05.219064 22729 hybrid_clock.cc:648] HybridClock initialized: now 1785977345219045 us; error 30 us; skew 500 ppm
I20260806 00:49:05.224086 22729 webserver.cc:460] Webserver started at http://127.15.147.190:38563/ using document root <none> and password file <none>
I20260806 00:49:05.224721 22729 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:05.224787 22729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:05.225026 22729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:05.227065 22729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "77bbe60b884946a498e4d9235f79b681"
format_stamp: "Formatted at 2026-08-06 00:49:05 on dist-test-slave-45dx"
server_key: "cba035e617df5a47f953dc9610033234"
server_key_iv: "8cb86ac239b2538373d42932255becc6"
server_key_version: "encryptionkey@0"
I20260806 00:49:05.227655 22729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "77bbe60b884946a498e4d9235f79b681"
format_stamp: "Formatted at 2026-08-06 00:49:05 on dist-test-slave-45dx"
server_key: "cba035e617df5a47f953dc9610033234"
server_key_iv: "8cb86ac239b2538373d42932255becc6"
server_key_version: "encryptionkey@0"
I20260806 00:49:05.231412 22729 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:49:05.233781 22751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:05.234895 22729 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 00:49:05.235021 22729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "77bbe60b884946a498e4d9235f79b681"
format_stamp: "Formatted at 2026-08-06 00:49:05 on dist-test-slave-45dx"
server_key: "cba035e617df5a47f953dc9610033234"
server_key_iv: "8cb86ac239b2538373d42932255becc6"
server_key_version: "encryptionkey@0"
I20260806 00:49:05.235128 22729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:05.239593 22729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:05.240361 22729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:05.240554 22729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:05.247422 22729 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:46717
I20260806 00:49:05.247445 22820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:46717 every 8 connection(s)
I20260806 00:49:05.248286 22729 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:49:05.251605 22821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:05.257079 22821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681: Bootstrap starting.
I20260806 00:49:05.257611 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 22729
I20260806 00:49:05.257750 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:49:05.258038 15950 external_mini_cluster.cc:1428] Setting key e18a1fcc3df5706dd379f6bc3a29181e
I20260806 00:49:05.259938 22821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:05.260866 22821 log.cc:824] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:05.263234 22821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681: No bootstrap required, opened a new log
I20260806 00:49:05.264137 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:05.259385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49562 (local address 127.15.147.190:46717)
0806 00:49:05.260186 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:05.260209 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:05.260254 (+ 45us) server_negotiation.cc:407] Connection header received
0806 00:49:05.261132 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:05.261156 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:05.261538 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:05.261897 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:05.262656 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:05.262662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:05.262676 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:05.262688 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:05.262758 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:05.262781 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:05.262901 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:05.262974 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 00:49:05.263234 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":98,"threads_started":1}
I20260806 00:49:05.267292 22821 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77bbe60b884946a498e4d9235f79b681" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46717 } }
I20260806 00:49:05.267503 22821 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:05.267575 22821 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77bbe60b884946a498e4d9235f79b681, State: Initialized, Role: FOLLOWER
I20260806 00:49:05.268055 22821 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77bbe60b884946a498e4d9235f79b681" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46717 } }
I20260806 00:49:05.268184 22821 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:49:05.268232 22821 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:49:05.268347 22821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:05.269239 22821 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77bbe60b884946a498e4d9235f79b681" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46717 } }
I20260806 00:49:05.269536 22821 leader_election.cc:302] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77bbe60b884946a498e4d9235f79b681; no voters:
I20260806 00:49:05.269783 22821 leader_election.cc:288] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:49:05.269924 22830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:05.270152 22830 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681 [term 1 LEADER]: Becoming Leader. State: Replica: 77bbe60b884946a498e4d9235f79b681, State: Running, Role: LEADER
I20260806 00:49:05.270473 22830 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77bbe60b884946a498e4d9235f79b681" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46717 } }
I20260806 00:49:05.271202 22821 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:49:05.272078 22833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77bbe60b884946a498e4d9235f79b681" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77bbe60b884946a498e4d9235f79b681" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46717 } } }
I20260806 00:49:05.272228 22833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:05.272181 22831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77bbe60b884946a498e4d9235f79b681. Latest consensus state: current_term: 1 leader_uuid: "77bbe60b884946a498e4d9235f79b681" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77bbe60b884946a498e4d9235f79b681" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46717 } } }
I20260806 00:49:05.272469 22831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681 [sys.catalog]: This master's current role is: LEADER
W20260806 00:49:05.276582 22855 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:49:05.276670 22855 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:49:05.276741 22841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:49:05.277428 22841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:49:05.282960 22841 catalog_manager.cc:1347] Generated new cluster ID: a785d84686954241924dcf0038aedf3c
I20260806 00:49:05.283027 22841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:49:05.290870 22841 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:49:05.291170 22841 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:49:05.297039 22841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681: Generated new TSK 0
I20260806 00:49:05.297827 22841 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:49:05.309995 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:46717
--builtin_ntp_servers=127.15.147.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 00:49:05.423874 22861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:05.424170 22861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:05.424211 22861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:05.428342 22861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:05.428404 22861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:05.428460 22861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:49:05.433364 22861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:46717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:05.434494 22861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:05.436278 22861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:05.446220 22866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:05.446208 22872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:05.446197 22867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:05.446501 22861 server_base.cc:1034] running on GCE node
I20260806 00:49:05.446936 22861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:05.447484 22861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:05.448642 22861 hybrid_clock.cc:648] HybridClock initialized: now 1785977345448622 us; error 29 us; skew 500 ppm
I20260806 00:49:05.450404 22861 webserver.cc:460] Webserver started at http://127.15.147.129:34897/ using document root <none> and password file <none>
I20260806 00:49:05.450968 22861 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:05.451030 22861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:05.451242 22861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:05.453318 22861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "5cd5e28f8119493e8df22144b40139d2"
format_stamp: "Formatted at 2026-08-06 00:49:05 on dist-test-slave-45dx"
server_key: "9ab26e1c29bbb75719f692fbc361da9c"
server_key_iv: "5326dd5b339fe6fb9c5e8ea0388b524a"
server_key_version: "encryptionkey@0"
I20260806 00:49:05.453898 22861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "5cd5e28f8119493e8df22144b40139d2"
format_stamp: "Formatted at 2026-08-06 00:49:05 on dist-test-slave-45dx"
server_key: "9ab26e1c29bbb75719f692fbc361da9c"
server_key_iv: "5326dd5b339fe6fb9c5e8ea0388b524a"
server_key_version: "encryptionkey@0"
I20260806 00:49:05.457551 22861 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:49:05.459944 22878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:05.461181 22861 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:49:05.461345 22861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "5cd5e28f8119493e8df22144b40139d2"
format_stamp: "Formatted at 2026-08-06 00:49:05 on dist-test-slave-45dx"
server_key: "9ab26e1c29bbb75719f692fbc361da9c"
server_key_iv: "5326dd5b339fe6fb9c5e8ea0388b524a"
server_key_version: "encryptionkey@0"
I20260806 00:49:05.461469 22861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:05.465957 22861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:05.466542 22861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:05.466727 22861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:05.467322 22861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:05.468204 22861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:05.468277 22861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:05.468376 22861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:05.468422 22861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:05.477739 22861 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:33631
I20260806 00:49:05.477833 23026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:33631 every 8 connection(s)
I20260806 00:49:05.478688 22861 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:49:05.485114 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:05.480562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:40429 (local address 127.15.147.190:46717)
0806 00:49:05.480707 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:05.480711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:05.481831 (+ 1120us) server_negotiation.cc:407] Connection header received
0806 00:49:05.483301 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:05.483304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:05.483351 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:05.483432 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:05.484634 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:05.484637 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:05.484640 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:05.484652 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:05.484672 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:05.484685 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:05.484781 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:05.484932 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 00:49:05.484977 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 00:49:05.485970 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:05.481001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:46717 (local address 127.15.147.129:40429)
0806 00:49:05.481668 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:05.481710 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:05.483091 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:05.483593 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:05.483600 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:05.483684 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:05.484344 (+ 660us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:49:05.484348 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:05.484408 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:49:05.484477 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:05.484766 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:05.484775 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:05.484781 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 00:49:05.484972 (+ 191us) client_negotiation.cc:240] Negotiation successful
0806 00:49:05.485204 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":117,"threads_started":1}
I20260806 00:49:05.487273 23028 heartbeater.cc:347] Connected to a master server at 127.15.147.190:46717
I20260806 00:49:05.487475 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 22861
I20260806 00:49:05.487519 23028 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:05.487617 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:49:05.487974 15950 external_mini_cluster.cc:1428] Setting key b098443603919d7d33dcb8d1e94bf0b6
I20260806 00:49:05.487972 23028 heartbeater.cc:511] Master 127.15.147.190:46717 requested a full tablet report, sending...
I20260806 00:49:05.489667 22770 ts_manager.cc:194] Registered new tserver with Master: 5cd5e28f8119493e8df22144b40139d2 (127.15.147.129:33631)
I20260806 00:49:05.490031 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:46717
--builtin_ntp_servers=127.15.147.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 00:49:05.603394 23034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:05.603694 23034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:05.603770 23034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:05.607805 23034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:05.607898 23034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:05.607987 23034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:49:05.612720 23034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:46717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:05.613773 23034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:05.615149 23034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:05.624671 23043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:05.624697 23047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:05.624711 23045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:05.624763 23034 server_base.cc:1034] running on GCE node
I20260806 00:49:05.625319 23034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:05.625936 23034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:05.627125 23034 hybrid_clock.cc:648] HybridClock initialized: now 1785977345627105 us; error 32 us; skew 500 ppm
I20260806 00:49:05.629034 23034 webserver.cc:460] Webserver started at http://127.15.147.130:42611/ using document root <none> and password file <none>
I20260806 00:49:05.629595 23034 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:05.629691 23034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:05.629935 23034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:05.631925 23034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "3ef8179e3d0241598dd9ae84d976c57c"
format_stamp: "Formatted at 2026-08-06 00:49:05 on dist-test-slave-45dx"
server_key: "cddcfed348a79765094c5fef360d46ca"
server_key_iv: "c61a88ed86434f83214c19c9c6c865af"
server_key_version: "encryptionkey@0"
I20260806 00:49:05.632551 23034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "3ef8179e3d0241598dd9ae84d976c57c"
format_stamp: "Formatted at 2026-08-06 00:49:05 on dist-test-slave-45dx"
server_key: "cddcfed348a79765094c5fef360d46ca"
server_key_iv: "c61a88ed86434f83214c19c9c6c865af"
server_key_version: "encryptionkey@0"
I20260806 00:49:05.636022 23034 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:49:05.638536 23054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:05.639748 23034 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:49:05.639896 23034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "3ef8179e3d0241598dd9ae84d976c57c"
format_stamp: "Formatted at 2026-08-06 00:49:05 on dist-test-slave-45dx"
server_key: "cddcfed348a79765094c5fef360d46ca"
server_key_iv: "c61a88ed86434f83214c19c9c6c865af"
server_key_version: "encryptionkey@0"
I20260806 00:49:05.640012 23034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:05.644485 23034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:05.645066 23034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:05.645246 23034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:05.645778 23034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:05.646651 23034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:05.646773 23034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:05.646891 23034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:05.646946 23034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:05.655694 23034 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:37719
I20260806 00:49:05.655716 23210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:37719 every 8 connection(s)
I20260806 00:49:05.656634 23034 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:49:05.656844 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 23034
I20260806 00:49:05.656960 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:49:05.657347 15950 external_mini_cluster.cc:1428] Setting key e7f6d4f9628dbd4f236675c51c276ce0
I20260806 00:49:05.659691 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:46717
--builtin_ntp_servers=127.15.147.148:39581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 00:49:05.663130 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:05.658533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:58383 (local address 127.15.147.190:46717)
0806 00:49:05.658669 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:05.658673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:05.659964 (+ 1291us) server_negotiation.cc:407] Connection header received
0806 00:49:05.661382 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:05.661385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:05.661426 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:05.661506 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:05.662643 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:05.662647 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:05.662650 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:05.662663 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:05.662689 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:05.662697 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:05.662787 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:05.662947 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 00:49:05.662993 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:49:05.664209 23213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:05.659077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:46717 (local address 127.15.147.130:58383)
0806 00:49:05.659813 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:05.659867 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:05.661177 (+ 1310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:05.661661 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:05.661667 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:05.661749 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:05.662392 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:49:05.662393 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:05.662433 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:49:05.662498 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:05.662784 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:05.662795 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:05.662804 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 00:49:05.662991 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 00:49:05.663226 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":138,"threads_started":1}
I20260806 00:49:05.665462 23211 heartbeater.cc:347] Connected to a master server at 127.15.147.190:46717
I20260806 00:49:05.665699 23211 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:05.666136 23211 heartbeater.cc:511] Master 127.15.147.190:46717 requested a full tablet report, sending...
I20260806 00:49:05.667269 22770 ts_manager.cc:194] Registered new tserver with Master: 3ef8179e3d0241598dd9ae84d976c57c (127.15.147.130:37719)
W20260806 00:49:05.778260 23214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:05.778565 23214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:05.778609 23214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:05.782665 23214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:05.782734 23214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:05.782794 23214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:49:05.787688 23214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:39581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:46717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:05.788774 23214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:05.790163 23214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:05.799975 23223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:05.800040 23222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:05.800016 23225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:05.800495 23214 server_base.cc:1034] running on GCE node
I20260806 00:49:05.800894 23214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:05.801505 23214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:05.802665 23214 hybrid_clock.cc:648] HybridClock initialized: now 1785977345802652 us; error 24 us; skew 500 ppm
I20260806 00:49:05.804558 23214 webserver.cc:460] Webserver started at http://127.15.147.131:38649/ using document root <none> and password file <none>
I20260806 00:49:05.805142 23214 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:05.805204 23214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:05.805449 23214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:05.807497 23214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "f0a9e8ef58ab4b538f84741a25f0a9c4"
format_stamp: "Formatted at 2026-08-06 00:49:05 on dist-test-slave-45dx"
server_key: "211218d214190bac4732c15d1d0163af"
server_key_iv: "ee673db75bace4ab261ef48b3066c659"
server_key_version: "encryptionkey@0"
I20260806 00:49:05.808061 23214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "f0a9e8ef58ab4b538f84741a25f0a9c4"
format_stamp: "Formatted at 2026-08-06 00:49:05 on dist-test-slave-45dx"
server_key: "211218d214190bac4732c15d1d0163af"
server_key_iv: "ee673db75bace4ab261ef48b3066c659"
server_key_version: "encryptionkey@0"
I20260806 00:49:05.811792 23214 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:49:05.814410 23233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:05.815658 23214 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:49:05.815788 23214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "f0a9e8ef58ab4b538f84741a25f0a9c4"
format_stamp: "Formatted at 2026-08-06 00:49:05 on dist-test-slave-45dx"
server_key: "211218d214190bac4732c15d1d0163af"
server_key_iv: "ee673db75bace4ab261ef48b3066c659"
server_key_version: "encryptionkey@0"
I20260806 00:49:05.815917 23214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:05.820744 23214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:05.821460 23214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:05.821620 23214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:05.822198 23214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:05.823065 23214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:05.823112 23214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:05.823149 23214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:05.823164 23214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:05.832183 23214 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:37317
I20260806 00:49:05.832208 23393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:37317 every 8 connection(s)
I20260806 00:49:05.833077 23214 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:49:05.838533 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 23214
I20260806 00:49:05.838660 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:49:05.839064 15950 external_mini_cluster.cc:1428] Setting key 0b3832f83e3321866d18eb77372b4985
I20260806 00:49:05.839320 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:05.834921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:50661 (local address 127.15.147.190:46717)
0806 00:49:05.835082 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:05.835087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:05.836164 (+ 1077us) server_negotiation.cc:407] Connection header received
0806 00:49:05.837459 (+ 1295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:05.837462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:05.837499 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:05.837571 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:05.838753 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:05.838756 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:05.838758 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:05.838769 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:05.838808 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:05.838818 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:05.838936 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:05.839142 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 00:49:05.839189 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 00:49:05.840401 23396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:05.835361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:46717 (local address 127.15.147.131:50661)
0806 00:49:05.836017 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:05.836057 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:05.837276 (+ 1219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:05.837727 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:05.837733 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:05.837814 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:05.838449 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:49:05.838451 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:05.838490 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:49:05.838566 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:05.838947 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:05.838958 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:05.838965 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 00:49:05.839165 (+ 200us) client_negotiation.cc:240] Negotiation successful
0806 00:49:05.839461 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":116,"threads_started":1}
I20260806 00:49:05.841460 23394 heartbeater.cc:347] Connected to a master server at 127.15.147.190:46717
I20260806 00:49:05.841638 23394 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:05.842007 23394 heartbeater.cc:511] Master 127.15.147.190:46717 requested a full tablet report, sending...
I20260806 00:49:05.842939 22771 ts_manager.cc:194] Registered new tserver with Master: f0a9e8ef58ab4b538f84741a25f0a9c4 (127.15.147.131:37317)
I20260806 00:49:05.852885 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:49:05.856556 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:05.854805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49570 (local address 127.15.147.190:46717)
0806 00:49:05.854987 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:05.854992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:05.855134 (+ 142us) server_negotiation.cc:407] Connection header received
0806 00:49:05.855300 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:05.855303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:05.855345 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:05.855429 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:05.856129 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:05.856133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:05.856136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:05.856148 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:05.856176 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:05.856183 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:05.856272 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:05.856380 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 00:49:05.856422 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 00:49:05.859364 22771 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:49:05.862504 22771 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:49:05.874704 23419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:05.872104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34432 (local address 127.15.147.130:37719)
0806 00:49:05.872628 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:05.872639 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:05.872662 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:49:05.872718 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:05.872723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:05.872891 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:05.873038 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:05.874047 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:05.874060 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:05.874071 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:05.874081 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:05.874119 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:05.874142 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:05.874266 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:05.874420 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 00:49:05.874507 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":121,"threads_started":1}
I20260806 00:49:05.875192 23412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:05.871725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:37719 (local address 127.0.0.1:34432)
0806 00:49:05.872287 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:05.872319 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:05.872433 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:05.873035 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:05.873041 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:05.873113 (+ 72us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:05.873800 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:49:05.873802 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:05.873822 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:49:05.873892 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:05.874238 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:05.874248 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:05.874254 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 00:49:05.874387 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 00:49:05.875046 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"spinlock_wait_cycles":5760,"thread_start_us":87,"threads_started":1}
I20260806 00:49:05.875193 23416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:05.871554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55298 (local address 127.15.147.129:33631)
0806 00:49:05.871981 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:05.871994 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:05.872540 (+ 546us) server_negotiation.cc:407] Connection header received
0806 00:49:05.872726 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:05.872735 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:05.872911 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:05.873041 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:05.874499 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:05.874504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:05.874514 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:05.874525 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:05.874564 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:05.874596 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:05.874703 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:05.874932 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 00:49:05.875005 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":122,"threads_started":1}
I20260806 00:49:05.875499 23417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:05.871123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:33631 (local address 127.0.0.1:55298)
0806 00:49:05.872242 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:05.872289 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:05.872433 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:05.873390 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:05.873395 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:05.873416 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:05.873878 (+ 462us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:49:05.873880 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:05.873887 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:49:05.873905 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:05.874749 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:05.874753 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:05.874758 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 00:49:05.874838 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 00:49:05.875271 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":957,"mutex_wait_us":369,"spinlock_wait_cycles":7808,"thread_start_us":143,"threads_started":1}
I20260806 00:49:05.875669 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:05.871158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38996 (local address 127.15.147.131:37317)
0806 00:49:05.871806 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:05.871821 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:05.872635 (+ 814us) server_negotiation.cc:407] Connection header received
0806 00:49:05.872756 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:05.872761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:05.872912 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:05.873041 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:05.874522 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:05.874527 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:05.874536 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:05.874549 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:05.874597 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:05.874629 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:05.874704 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:05.875147 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 00:49:05.875217 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":236,"threads_started":1}
I20260806 00:49:05.876045 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:05.870976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:37317 (local address 127.0.0.1:38996)
0806 00:49:05.872468 (+ 1492us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:05.872512 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:05.872655 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:05.873816 (+ 1161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:05.873820 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:05.873839 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:05.874308 (+ 469us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:49:05.874310 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:05.874322 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:49:05.874350 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:05.875015 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:05.875018 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:05.875021 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 00:49:05.875874 (+ 853us) client_negotiation.cc:240] Negotiation successful
0806 00:49:05.875912 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":910,"thread_start_us":70,"threads_started":1}
I20260806 00:49:05.878378 22941 tablet_service.cc:1467] Processing CreateTablet for tablet e434c4c91d79469ca368c2acc558d0cb (DEFAULT_TABLE table=test-table [id=8779da246f6c42049e97731ba5b9712d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:05.878376 23117 tablet_service.cc:1467] Processing CreateTablet for tablet e434c4c91d79469ca368c2acc558d0cb (DEFAULT_TABLE table=test-table [id=8779da246f6c42049e97731ba5b9712d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:05.878609 23301 tablet_service.cc:1467] Processing CreateTablet for tablet e434c4c91d79469ca368c2acc558d0cb (DEFAULT_TABLE table=test-table [id=8779da246f6c42049e97731ba5b9712d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:05.879415 22941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e434c4c91d79469ca368c2acc558d0cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:05.879451 23117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e434c4c91d79469ca368c2acc558d0cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:05.879459 23301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e434c4c91d79469ca368c2acc558d0cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:05.885643 23420 tablet_bootstrap.cc:492] T e434c4c91d79469ca368c2acc558d0cb P 5cd5e28f8119493e8df22144b40139d2: Bootstrap starting.
I20260806 00:49:05.887039 23423 tablet_bootstrap.cc:492] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c: Bootstrap starting.
I20260806 00:49:05.887447 23422 tablet_bootstrap.cc:492] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4: Bootstrap starting.
I20260806 00:49:05.888691 23420 tablet_bootstrap.cc:654] T e434c4c91d79469ca368c2acc558d0cb P 5cd5e28f8119493e8df22144b40139d2: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:05.889183 23423 tablet_bootstrap.cc:654] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:05.889426 23422 tablet_bootstrap.cc:654] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:05.889530 23420 log.cc:824] T e434c4c91d79469ca368c2acc558d0cb P 5cd5e28f8119493e8df22144b40139d2: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:05.890085 23423 log.cc:824] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:05.890187 23422 log.cc:824] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:05.891808 23420 tablet_bootstrap.cc:492] T e434c4c91d79469ca368c2acc558d0cb P 5cd5e28f8119493e8df22144b40139d2: No bootstrap required, opened a new log
I20260806 00:49:05.891834 23422 tablet_bootstrap.cc:492] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4: No bootstrap required, opened a new log
I20260806 00:49:05.892053 23422 ts_tablet_manager.cc:1397] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260806 00:49:05.892055 23420 ts_tablet_manager.cc:1397] T e434c4c91d79469ca368c2acc558d0cb P 5cd5e28f8119493e8df22144b40139d2: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 00:49:05.892076 23423 tablet_bootstrap.cc:492] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c: No bootstrap required, opened a new log
I20260806 00:49:05.892230 23423 ts_tablet_manager.cc:1397] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:49:05.895769 23422 raft_consensus.cc:348] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a9e8ef58ab4b538f84741a25f0a9c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37317 } } peers { permanent_uuid: "5cd5e28f8119493e8df22144b40139d2" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33631 } } peers { permanent_uuid: "3ef8179e3d0241598dd9ae84d976c57c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37719 } }
I20260806 00:49:05.896201 23422 raft_consensus.cc:374] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:05.896270 23422 raft_consensus.cc:729] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0a9e8ef58ab4b538f84741a25f0a9c4, State: Initialized, Role: FOLLOWER
I20260806 00:49:05.896817 23422 consensus_queue.cc:260] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a9e8ef58ab4b538f84741a25f0a9c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37317 } } peers { permanent_uuid: "5cd5e28f8119493e8df22144b40139d2" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33631 } } peers { permanent_uuid: "3ef8179e3d0241598dd9ae84d976c57c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37719 } }
I20260806 00:49:05.896987 23420 raft_consensus.cc:348] T e434c4c91d79469ca368c2acc558d0cb P 5cd5e28f8119493e8df22144b40139d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a9e8ef58ab4b538f84741a25f0a9c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37317 } } peers { permanent_uuid: "5cd5e28f8119493e8df22144b40139d2" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33631 } } peers { permanent_uuid: "3ef8179e3d0241598dd9ae84d976c57c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37719 } }
I20260806 00:49:05.896987 23423 raft_consensus.cc:348] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a9e8ef58ab4b538f84741a25f0a9c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37317 } } peers { permanent_uuid: "5cd5e28f8119493e8df22144b40139d2" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33631 } } peers { permanent_uuid: "3ef8179e3d0241598dd9ae84d976c57c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37719 } }
I20260806 00:49:05.897272 23420 raft_consensus.cc:374] T e434c4c91d79469ca368c2acc558d0cb P 5cd5e28f8119493e8df22144b40139d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:05.897272 23423 raft_consensus.cc:374] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:05.897351 23423 raft_consensus.cc:729] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef8179e3d0241598dd9ae84d976c57c, State: Initialized, Role: FOLLOWER
I20260806 00:49:05.897359 23420 raft_consensus.cc:729] T e434c4c91d79469ca368c2acc558d0cb P 5cd5e28f8119493e8df22144b40139d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd5e28f8119493e8df22144b40139d2, State: Initialized, Role: FOLLOWER
I20260806 00:49:05.897768 23394 heartbeater.cc:503] Master 127.15.147.190:46717 was elected leader, sending a full tablet report...
I20260806 00:49:05.897807 23423 consensus_queue.cc:260] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a9e8ef58ab4b538f84741a25f0a9c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37317 } } peers { permanent_uuid: "5cd5e28f8119493e8df22144b40139d2" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33631 } } peers { permanent_uuid: "3ef8179e3d0241598dd9ae84d976c57c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37719 } }
I20260806 00:49:05.897852 23420 consensus_queue.cc:260] T e434c4c91d79469ca368c2acc558d0cb P 5cd5e28f8119493e8df22144b40139d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a9e8ef58ab4b538f84741a25f0a9c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37317 } } peers { permanent_uuid: "5cd5e28f8119493e8df22144b40139d2" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33631 } } peers { permanent_uuid: "3ef8179e3d0241598dd9ae84d976c57c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37719 } }
I20260806 00:49:05.898026 23422 ts_tablet_manager.cc:1428] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:49:05.898751 23423 ts_tablet_manager.cc:1428] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 00:49:05.898751 23420 ts_tablet_manager.cc:1428] T e434c4c91d79469ca368c2acc558d0cb P 5cd5e28f8119493e8df22144b40139d2: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 00:49:05.899305 23028 heartbeater.cc:503] Master 127.15.147.190:46717 was elected leader, sending a full tablet report...
I20260806 00:49:05.900334 23211 heartbeater.cc:503] Master 127.15.147.190:46717 was elected leader, sending a full tablet report...
W20260806 00:49:05.907723 23212 tablet.cc:2369] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:49:05.980170 23029 tablet.cc:2369] T e434c4c91d79469ca368c2acc558d0cb P 5cd5e28f8119493e8df22144b40139d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:49:06.084162 23395 tablet.cc:2369] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:49:06.134035 23427 raft_consensus.cc:482] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:49:06.134323 23427 raft_consensus.cc:504] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a9e8ef58ab4b538f84741a25f0a9c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37317 } } peers { permanent_uuid: "5cd5e28f8119493e8df22144b40139d2" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33631 } } peers { permanent_uuid: "3ef8179e3d0241598dd9ae84d976c57c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37719 } }
I20260806 00:49:06.135690 23427 leader_election.cc:288] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd5e28f8119493e8df22144b40139d2 (127.15.147.129:33631), 3ef8179e3d0241598dd9ae84d976c57c (127.15.147.130:37719)
I20260806 00:49:06.138005 23396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:06.135944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:33631 (local address 127.15.147.131:39001)
0806 00:49:06.136120 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:06.136137 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:06.136232 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:06.136596 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:06.136599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:06.136617 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:06.137186 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:49:06.137188 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:06.137199 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:49:06.137249 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:06.137656 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:06.137659 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:06.137663 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 00:49:06.137768 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 00:49:06.137812 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 00:49:06.138060 23416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:06.136179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:39001 (local address 127.15.147.129:33631)
0806 00:49:06.136345 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:06.136349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:06.136379 (+ 30us) server_negotiation.cc:407] Connection header received
0806 00:49:06.136420 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:06.136423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:06.136457 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:06.136567 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:06.137463 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:06.137468 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:06.137473 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:06.137493 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:06.137519 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:06.137530 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:06.137651 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:06.137807 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 00:49:06.137865 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:49:06.138351 23419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:06.136175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:46949 (local address 127.15.147.130:37719)
0806 00:49:06.136347 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:06.136354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:06.136701 (+ 347us) server_negotiation.cc:407] Connection header received
0806 00:49:06.136853 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:06.136856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:06.136908 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:06.137002 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:06.137679 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:06.137684 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:06.137686 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:06.137701 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:06.137729 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:06.137742 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:06.137831 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:06.138186 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 00:49:06.138235 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:49:06.138720 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:06.136039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:37719 (local address 127.15.147.131:46949)
0806 00:49:06.136494 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:06.136505 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:06.136724 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:06.137022 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:06.137025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:06.137040 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:06.137502 (+ 462us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:49:06.137504 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:06.137516 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:49:06.137545 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:06.137866 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:06.137869 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:06.137873 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 00:49:06.138555 (+ 682us) client_negotiation.cc:240] Negotiation successful
0806 00:49:06.138597 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":139,"threads_started":1}
I20260806 00:49:06.138809 22965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e434c4c91d79469ca368c2acc558d0cb" candidate_uuid: "f0a9e8ef58ab4b538f84741a25f0a9c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd5e28f8119493e8df22144b40139d2" is_pre_election: true
I20260806 00:49:06.139214 22965 raft_consensus.cc:2454] T e434c4c91d79469ca368c2acc558d0cb P 5cd5e28f8119493e8df22144b40139d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0a9e8ef58ab4b538f84741a25f0a9c4 in term 0.
I20260806 00:49:06.139443 23140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e434c4c91d79469ca368c2acc558d0cb" candidate_uuid: "f0a9e8ef58ab4b538f84741a25f0a9c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef8179e3d0241598dd9ae84d976c57c" is_pre_election: true
I20260806 00:49:06.139734 23235 leader_election.cc:302] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd5e28f8119493e8df22144b40139d2, f0a9e8ef58ab4b538f84741a25f0a9c4; no voters:
I20260806 00:49:06.139801 23140 raft_consensus.cc:2454] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0a9e8ef58ab4b538f84741a25f0a9c4 in term 0.
I20260806 00:49:06.140020 23427 raft_consensus.cc:2781] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:49:06.140115 23427 raft_consensus.cc:482] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:49:06.140180 23427 raft_consensus.cc:3037] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:06.141425 23427 raft_consensus.cc:504] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a9e8ef58ab4b538f84741a25f0a9c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37317 } } peers { permanent_uuid: "5cd5e28f8119493e8df22144b40139d2" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33631 } } peers { permanent_uuid: "3ef8179e3d0241598dd9ae84d976c57c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37719 } }
I20260806 00:49:06.141875 23427 leader_election.cc:288] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [CANDIDATE]: Term 1 election: Requested vote from peers 5cd5e28f8119493e8df22144b40139d2 (127.15.147.129:33631), 3ef8179e3d0241598dd9ae84d976c57c (127.15.147.130:37719)
I20260806 00:49:06.142215 22965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e434c4c91d79469ca368c2acc558d0cb" candidate_uuid: "f0a9e8ef58ab4b538f84741a25f0a9c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd5e28f8119493e8df22144b40139d2"
I20260806 00:49:06.142271 23140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e434c4c91d79469ca368c2acc558d0cb" candidate_uuid: "f0a9e8ef58ab4b538f84741a25f0a9c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef8179e3d0241598dd9ae84d976c57c"
I20260806 00:49:06.142349 22965 raft_consensus.cc:3037] T e434c4c91d79469ca368c2acc558d0cb P 5cd5e28f8119493e8df22144b40139d2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:06.142397 23140 raft_consensus.cc:3037] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:06.143783 22965 raft_consensus.cc:2454] T e434c4c91d79469ca368c2acc558d0cb P 5cd5e28f8119493e8df22144b40139d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0a9e8ef58ab4b538f84741a25f0a9c4 in term 1.
I20260806 00:49:06.143783 23140 raft_consensus.cc:2454] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0a9e8ef58ab4b538f84741a25f0a9c4 in term 1.
I20260806 00:49:06.144120 23235 leader_election.cc:302] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cd5e28f8119493e8df22144b40139d2, f0a9e8ef58ab4b538f84741a25f0a9c4; no voters:
I20260806 00:49:06.144403 23427 raft_consensus.cc:2781] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:06.144695 23427 raft_consensus.cc:686] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [term 1 LEADER]: Becoming Leader. State: Replica: f0a9e8ef58ab4b538f84741a25f0a9c4, State: Running, Role: LEADER
I20260806 00:49:06.145099 23427 consensus_queue.cc:237] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a9e8ef58ab4b538f84741a25f0a9c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37317 } } peers { permanent_uuid: "5cd5e28f8119493e8df22144b40139d2" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33631 } } peers { permanent_uuid: "3ef8179e3d0241598dd9ae84d976c57c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37719 } }
I20260806 00:49:06.148523 22771 catalog_manager.cc:5495] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0a9e8ef58ab4b538f84741a25f0a9c4 (127.15.147.131). New cstate: current_term: 1 leader_uuid: "f0a9e8ef58ab4b538f84741a25f0a9c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a9e8ef58ab4b538f84741a25f0a9c4" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd5e28f8119493e8df22144b40139d2" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef8179e3d0241598dd9ae84d976c57c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37719 } health_report { overall_health: UNKNOWN } } }
I20260806 00:49:06.166915 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:06.165290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39004 (local address 127.15.147.131:37317)
0806 00:49:06.165423 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:06.165427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:06.165459 (+ 32us) server_negotiation.cc:407] Connection header received
0806 00:49:06.165639 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:06.165641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:06.165699 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:06.165798 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:06.166531 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:06.166534 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:06.166536 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:06.166546 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:06.166566 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:06.166574 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:06.166647 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:06.166704 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 00:49:06.166736 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 00:49:06.172693 23140 raft_consensus.cc:1261] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c [term 1 FOLLOWER]: Refusing update from remote peer f0a9e8ef58ab4b538f84741a25f0a9c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:06.172701 22965 raft_consensus.cc:1261] T e434c4c91d79469ca368c2acc558d0cb P 5cd5e28f8119493e8df22144b40139d2 [term 1 FOLLOWER]: Refusing update from remote peer f0a9e8ef58ab4b538f84741a25f0a9c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:06.173445 23427 consensus_queue.cc:1035] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd5e28f8119493e8df22144b40139d2" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:06.173659 23431 consensus_queue.cc:1035] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef8179e3d0241598dd9ae84d976c57c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:06.180148 23434 mvcc.cc:205] Tried to move back new op lower bound from 7315363209915826176 to 7315363209813479424. Current Snapshot: MvccSnapshot[applied={T|T < 7315363209915826176}]
I20260806 00:49:06.183279 23436 mvcc.cc:205] Tried to move back new op lower bound from 7315363209915826176 to 7315363209813479424. Current Snapshot: MvccSnapshot[applied={T|T < 7315363209915826176 or (T in {7315363209915826176})}]
I20260806 00:49:06.189024 22770 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:49:06.189213 22770 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:49:06.191857 22770 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 77bbe60b884946a498e4d9235f79b681: Sending DeleteTablet for 3 replicas of tablet e434c4c91d79469ca368c2acc558d0cb
I20260806 00:49:06.192809 23117 tablet_service.cc:1514] Processing DeleteTablet for tablet e434c4c91d79469ca368c2acc558d0cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:06 UTC) from {username='slave'} at 127.0.0.1:34432
I20260806 00:49:06.192807 22941 tablet_service.cc:1514] Processing DeleteTablet for tablet e434c4c91d79469ca368c2acc558d0cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:06 UTC) from {username='slave'} at 127.0.0.1:55298
I20260806 00:49:06.193471 23445 tablet_replica.cc:330] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c: stopping tablet replica
I20260806 00:49:06.193728 23301 tablet_service.cc:1514] Processing DeleteTablet for tablet e434c4c91d79469ca368c2acc558d0cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:06 UTC) from {username='slave'} at 127.0.0.1:38996
I20260806 00:49:06.193796 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 22861
I20260806 00:49:06.194049 23445 raft_consensus.cc:2229] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:06.194809 23446 tablet_replica.cc:330] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4: stopping tablet replica
I20260806 00:49:06.195037 23445 raft_consensus.cc:2258] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:06.195122 23446 raft_consensus.cc:2229] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:49:06.198841 23446 raft_consensus.cc:2258] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:06.199468 23445 ts_tablet_manager.cc:1905] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:06.203246 23445 ts_tablet_manager.cc:1918] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:49:06.203441 23445 log.cc:1196] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/e434c4c91d79469ca368c2acc558d0cb
I20260806 00:49:06.203859 23445 ts_tablet_manager.cc:1939] T e434c4c91d79469ca368c2acc558d0cb P 3ef8179e3d0241598dd9ae84d976c57c: Deleting consensus metadata
I20260806 00:49:06.204007 23446 ts_tablet_manager.cc:1905] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:06.205499 22753 catalog_manager.cc:4841] TS 3ef8179e3d0241598dd9ae84d976c57c (127.15.147.130:37719): tablet e434c4c91d79469ca368c2acc558d0cb (table test-table [id=8779da246f6c42049e97731ba5b9712d]) successfully deleted
I20260806 00:49:06.206947 23446 ts_tablet_manager.cc:1918] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:49:06.207049 23446 log.cc:1196] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/e434c4c91d79469ca368c2acc558d0cb
I20260806 00:49:06.207389 23446 ts_tablet_manager.cc:1939] T e434c4c91d79469ca368c2acc558d0cb P f0a9e8ef58ab4b538f84741a25f0a9c4: Deleting consensus metadata
I20260806 00:49:06.208655 22753 catalog_manager.cc:4841] TS f0a9e8ef58ab4b538f84741a25f0a9c4 (127.15.147.131:37317): tablet e434c4c91d79469ca368c2acc558d0cb (table test-table [id=8779da246f6c42049e97731ba5b9712d]) successfully deleted
W20260806 00:49:06.210338 22752 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.147.129:33631 (error 108)
I20260806 00:49:06.210491 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 23034
I20260806 00:49:06.214100 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:06.213787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:33631 (local address 127.0.0.1:55314)
0806 00:49:06.213937 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:06.214020 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 00:49:06.214355 22752 catalog_manager.cc:4584] TS 5cd5e28f8119493e8df22144b40139d2 (127.15.147.129:33631): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e434c4c91d79469ca368c2acc558d0cb: Network error: Client connection negotiation failed: client connection to 127.15.147.129:33631: connect: Connection refused (error 111)
I20260806 00:49:06.222678 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 23214
I20260806 00:49:06.235668 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 22729
2026-08-06T00:49:06Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1193 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 00:49:06.266786 15950 mini_oidc.cc:152] Starting JWKS server
I20260806 00:49:06.268774 15950 webserver.cc:460] Webserver started at https://127.0.0.1:41727/ using document root <none> and password file <none>
I20260806 00:49:06.268988 15950 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 00:49:06.269282 15950 webserver.cc:460] Webserver started at http://127.0.0.1:41359/ using document root <none> and password file <none>
2026-08-06T00:49:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:49:06Z Disabled control of system clock
I20260806 00:49:06.281625 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:40575
--webserver_interface=127.15.147.190
--webserver_port=0
--builtin_ntp_servers=127.15.147.148:41331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:40575
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41727/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/testchainca.cert with env {}
W20260806 00:49:06.397324 23453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:06.397611 23453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:06.397651 23453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:06.401743 23453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:49:06.401830 23453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:06.401846 23453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:06.401863 23453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:49:06.401880 23453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 00:49:06.401894 23453 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 00:49:06.401908 23453 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41727/jwks.json
I20260806 00:49:06.407351 23453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:41331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:40575
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:40575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41727/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:06.408454 23453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:06.409432 23453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:06.418170 23468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:06.418170 23466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:06.418208 23465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:06.418804 23453 server_base.cc:1034] running on GCE node
I20260806 00:49:06.419395 23453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:06.420619 23453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:06.421867 23453 hybrid_clock.cc:648] HybridClock initialized: now 1785977346421838 us; error 43 us; skew 500 ppm
I20260806 00:49:06.423732 23453 webserver.cc:460] Webserver started at http://127.15.147.190:46445/ using document root <none> and password file <none>
I20260806 00:49:06.424428 23453 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:06.424527 23453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:06.424749 23453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:06.426970 23453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "d0ebd74b305d400e94a26cbd5ffe440e"
format_stamp: "Formatted at 2026-08-06 00:49:06 on dist-test-slave-45dx"
server_key: "7e30a2f05f8752cdbe74e7e08b30abf4"
server_key_iv: "eae54754552e659fef2e71a208de8677"
server_key_version: "encryptionkey@0"
I20260806 00:49:06.427623 23453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "d0ebd74b305d400e94a26cbd5ffe440e"
format_stamp: "Formatted at 2026-08-06 00:49:06 on dist-test-slave-45dx"
server_key: "7e30a2f05f8752cdbe74e7e08b30abf4"
server_key_iv: "eae54754552e659fef2e71a208de8677"
server_key_version: "encryptionkey@0"
I20260806 00:49:06.431363 23453 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:49:06.433816 23476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:06.435086 23453 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:49:06.435254 23453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "d0ebd74b305d400e94a26cbd5ffe440e"
format_stamp: "Formatted at 2026-08-06 00:49:06 on dist-test-slave-45dx"
server_key: "7e30a2f05f8752cdbe74e7e08b30abf4"
server_key_iv: "eae54754552e659fef2e71a208de8677"
server_key_version: "encryptionkey@0"
I20260806 00:49:06.435374 23453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:06.450582 23453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:06.451272 23453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:06.451486 23453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:06.458811 23453 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:40575
I20260806 00:49:06.458818 23548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:40575 every 8 connection(s)
I20260806 00:49:06.459795 23453 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:49:06.463191 23550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:06.468233 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 23453
I20260806 00:49:06.468405 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:49:06.468705 15950 external_mini_cluster.cc:1428] Setting key 541a88da75ad78e7945ecdcaa11a81de
I20260806 00:49:06.468987 23550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e: Bootstrap starting.
I20260806 00:49:06.471868 23550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:06.472817 23550 log.cc:824] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:06.475001 23550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e: No bootstrap required, opened a new log
I20260806 00:49:06.479555 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:06.470132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44196 (local address 127.15.147.190:40575)
0806 00:49:06.470829 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:06.470866 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:06.470910 (+ 44us) server_negotiation.cc:407] Connection header received
0806 00:49:06.471847 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:06.471872 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:06.472436 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:06.472805 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:06.473266 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:06.474636 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:06.475615 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:06.476274 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:06.476482 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:06.477282 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:06.477289 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:06.477304 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:06.477819 (+ 515us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:06.477890 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:06.477919 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:06.478025 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:06.478172 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 00:49:06.478520 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":107,"threads_started":1}
I20260806 00:49:06.479568 23550 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ebd74b305d400e94a26cbd5ffe440e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40575 } }
I20260806 00:49:06.479921 23550 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:06.480006 23550 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0ebd74b305d400e94a26cbd5ffe440e, State: Initialized, Role: FOLLOWER
I20260806 00:49:06.480497 23550 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ebd74b305d400e94a26cbd5ffe440e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40575 } }
I20260806 00:49:06.480662 23550 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:49:06.480738 23550 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:49:06.480852 23550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:06.481974 23550 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ebd74b305d400e94a26cbd5ffe440e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40575 } }
I20260806 00:49:06.482339 23550 leader_election.cc:302] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0ebd74b305d400e94a26cbd5ffe440e; no voters:
I20260806 00:49:06.482704 23550 leader_election.cc:288] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:49:06.482731 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:49:06.482805 23555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:06.483016 23555 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e [term 1 LEADER]: Becoming Leader. State: Replica: d0ebd74b305d400e94a26cbd5ffe440e, State: Running, Role: LEADER
I20260806 00:49:06.483338 23555 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ebd74b305d400e94a26cbd5ffe440e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40575 } }
I20260806 00:49:06.484150 23550 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:49:06.484615 23555 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0ebd74b305d400e94a26cbd5ffe440e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ebd74b305d400e94a26cbd5ffe440e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40575 } } }
I20260806 00:49:06.484745 23555 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:06.485172 23561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:49:06.485177 23557 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0ebd74b305d400e94a26cbd5ffe440e. Latest consensus state: current_term: 1 leader_uuid: "d0ebd74b305d400e94a26cbd5ffe440e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ebd74b305d400e94a26cbd5ffe440e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40575 } } }
I20260806 00:49:06.485446 23557 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:06.488868 23561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:49:06.495566 23561 catalog_manager.cc:1347] Generated new cluster ID: 9c22a272250c4246af60aa576d6ad9fe
I20260806 00:49:06.495682 23561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:49:06.509039 23561 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:49:06.510075 23561 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:49:06.534137 23561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0ebd74b305d400e94a26cbd5ffe440e: Generated new TSK 0
I20260806 00:49:06.535012 23561 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 00:49:06.561656 23592 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 00:49:06.561862 23592 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 00:49:06.562369 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:06.545300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44204 (local address 127.15.147.190:40575)
0806 00:49:06.545453 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:06.545457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:06.545680 (+ 223us) server_negotiation.cc:407] Connection header received
0806 00:49:06.545914 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:06.545916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:06.545962 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:06.546042 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0806 00:49:06.546446 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:06.546981 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:06.548211 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:06.548497 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:06.548577 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:06.548981 (+ 404us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 00:49:06.562029 (+ 13048us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41727/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 00:49:06.562254 (+ 225us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44204: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41727/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 00:49:06.562484 23553 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44204: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41727/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 00:49:06.564133 23591 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:06.545212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40575 (local address 127.0.0.1:44204)
0806 00:49:06.545585 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:06.545602 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:06.545739 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:06.546092 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:06.546097 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:06.546108 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0806 00:49:06.546318 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:06.546328 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:06.547098 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:06.547102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:06.548091 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:06.548101 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:06.548611 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:06.548615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:06.548725 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:06.548784 (+ 59us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 00:49:06.562249 (+ 13465us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41727/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 00:49:06.562509 (+ 260us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.147.190:40575: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41727/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":132,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.15.147.190:40575: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41727/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 00:49:06.565032 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 23453
2026-08-06T00:49:06Z chronyd exiting
I20260806 00:49:06.612287 15950 test_util.cc:175] -----------------------------------------------
I20260806 00:49:06.612445 15950 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785977317010501-15950-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (348 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 00:49:06.613685 15950 mini_oidc.cc:152] Starting JWKS server
I20260806 00:49:06.614616 15950 webserver.cc:460] Webserver started at https://127.0.0.1:33947/ using document root <none> and password file <none>
I20260806 00:49:06.614727 15950 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 00:49:06.615041 15950 webserver.cc:460] Webserver started at http://127.0.0.1:40323/ using document root <none> and password file <none>
2026-08-06T00:49:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:49:06Z Disabled control of system clock
I20260806 00:49:06.628571 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:39379
--webserver_interface=127.15.147.190
--webserver_port=0
--builtin_ntp_servers=127.15.147.148:33971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:39379
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33947/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/testchainca.cert with env {}
W20260806 00:49:06.740028 23602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:06.740372 23602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:06.740482 23602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:06.744477 23602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:49:06.744553 23602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:06.744580 23602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:06.744642 23602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:49:06.744691 23602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 00:49:06.744736 23602 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 00:49:06.744764 23602 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33947/jwks.json
I20260806 00:49:06.750007 23602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:33971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:39379
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:39379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33947/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:06.751024 23602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:06.752008 23602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:06.760368 23614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:06.760375 23609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:06.760521 23602 server_base.cc:1034] running on GCE node
W20260806 00:49:06.760675 23610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:06.761235 23602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:06.762253 23602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:06.763427 23602 hybrid_clock.cc:648] HybridClock initialized: now 1785977346763407 us; error 31 us; skew 500 ppm
I20260806 00:49:06.765334 23602 webserver.cc:460] Webserver started at http://127.15.147.190:37433/ using document root <none> and password file <none>
I20260806 00:49:06.765906 23602 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:06.765969 23602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:06.766167 23602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:06.768078 23602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "4fc53e887171406286e767f6eabb8c81"
format_stamp: "Formatted at 2026-08-06 00:49:06 on dist-test-slave-45dx"
server_key: "a43047090f0506f1fe544b3a42c0b70c"
server_key_iv: "ceb7e0519d3bab52d32f7e3f863de23b"
server_key_version: "encryptionkey@0"
I20260806 00:49:06.768684 23602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "4fc53e887171406286e767f6eabb8c81"
format_stamp: "Formatted at 2026-08-06 00:49:06 on dist-test-slave-45dx"
server_key: "a43047090f0506f1fe544b3a42c0b70c"
server_key_iv: "ceb7e0519d3bab52d32f7e3f863de23b"
server_key_version: "encryptionkey@0"
I20260806 00:49:06.772411 23602 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:49:06.774698 23621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:06.776083 23602 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:49:06.776225 23602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "4fc53e887171406286e767f6eabb8c81"
format_stamp: "Formatted at 2026-08-06 00:49:06 on dist-test-slave-45dx"
server_key: "a43047090f0506f1fe544b3a42c0b70c"
server_key_iv: "ceb7e0519d3bab52d32f7e3f863de23b"
server_key_version: "encryptionkey@0"
I20260806 00:49:06.776363 23602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:06.793648 23602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:06.794440 23602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:06.794656 23602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:06.801673 23602 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:39379
I20260806 00:49:06.801685 23697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:39379 every 8 connection(s)
I20260806 00:49:06.802620 23602 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:49:06.805652 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 23602
I20260806 00:49:06.805770 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:49:06.805974 23699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:06.806133 15950 external_mini_cluster.cc:1428] Setting key 8e1a6d23252f2cdbd47e611068ea9d26
I20260806 00:49:06.812673 23699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81: Bootstrap starting.
I20260806 00:49:06.815167 23699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:06.815528 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:06.807412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40824 (local address 127.15.147.190:39379)
0806 00:49:06.808015 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:06.808029 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:06.808062 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:49:06.809117 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:06.809137 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:06.809501 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:06.809759 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:06.810078 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:06.811045 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:06.812253 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:06.812834 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:06.812950 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:06.813812 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:06.813818 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:06.813829 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:06.814176 (+ 347us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:06.814227 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:06.814246 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:06.814323 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:06.814487 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 00:49:06.814796 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":143,"threads_started":1}
I20260806 00:49:06.816015 23699 log.cc:824] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:06.818157 23699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81: No bootstrap required, opened a new log
I20260806 00:49:06.818403 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:49:06.822083 23699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc53e887171406286e767f6eabb8c81" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39379 } }
I20260806 00:49:06.822284 23699 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:06.822319 23699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fc53e887171406286e767f6eabb8c81, State: Initialized, Role: FOLLOWER
I20260806 00:49:06.822700 23699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc53e887171406286e767f6eabb8c81" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39379 } }
I20260806 00:49:06.822806 23699 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:49:06.822849 23699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:49:06.822919 23699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:06.823679 23699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc53e887171406286e767f6eabb8c81" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39379 } }
I20260806 00:49:06.823951 23699 leader_election.cc:302] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fc53e887171406286e767f6eabb8c81; no voters:
I20260806 00:49:06.824180 23699 leader_election.cc:288] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:49:06.824337 23704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:06.824554 23704 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81 [term 1 LEADER]: Becoming Leader. State: Replica: 4fc53e887171406286e767f6eabb8c81, State: Running, Role: LEADER
I20260806 00:49:06.824878 23704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc53e887171406286e767f6eabb8c81" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39379 } }
I20260806 00:49:06.825354 23699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:49:06.826035 23706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fc53e887171406286e767f6eabb8c81. Latest consensus state: current_term: 1 leader_uuid: "4fc53e887171406286e767f6eabb8c81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc53e887171406286e767f6eabb8c81" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39379 } } }
I20260806 00:49:06.826169 23706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:06.826481 23705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fc53e887171406286e767f6eabb8c81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc53e887171406286e767f6eabb8c81" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39379 } } }
I20260806 00:49:06.826591 23705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:06.826637 23714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:49:06.829690 23714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:49:06.834965 23714 catalog_manager.cc:1347] Generated new cluster ID: 20e18c967c00486480326eb4bfd1c08e
I20260806 00:49:06.835041 23714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:49:06.854981 23714 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:49:06.855947 23714 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:49:06.862811 23714 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fc53e887171406286e767f6eabb8c81: Generated new TSK 0
I20260806 00:49:06.863459 23714 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 00:49:06.891991 23731 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 00:49:06.892098 23731 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 00:49:06.892867 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:06.876137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40830 (local address 127.15.147.190:39379)
0806 00:49:06.876281 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:06.876285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:06.876458 (+ 173us) server_negotiation.cc:407] Connection header received
0806 00:49:06.876596 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:06.876599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:06.876661 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:06.876781 (+ 120us) server_negotiation.cc:226] Negotiated authn=JWT
0806 00:49:06.877069 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:06.877905 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:06.878765 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:06.879125 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:06.879232 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:06.879606 (+ 374us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 00:49:06.892418 (+ 12812us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33947/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 00:49:06.892719 (+ 301us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40830: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33947/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 00:49:06.892951 23702 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40830: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33947/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 00:49:06.892966 23730 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:06.876027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:39379 (local address 127.0.0.1:40830)
0806 00:49:06.876365 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:06.876378 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:06.876475 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:06.876819 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:06.876822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:06.876830 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 00:49:06.876950 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:06.876955 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:06.878025 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:06.878027 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:06.878653 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:06.878661 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:06.879302 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:06.879307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:06.879392 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:06.879420 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 00:49:06.892624 (+ 13204us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33947/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 00:49:06.892793 (+ 169us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.147.190:39379: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33947/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":104,"threads_started":1}
I20260806 00:49:06.893642 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 23602
2026-08-06T00:49:06Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (327 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 00:49:06.941502 15950 mini_oidc.cc:152] Starting JWKS server
I20260806 00:49:06.942430 15950 webserver.cc:460] Webserver started at https://127.0.0.1:44915/ using document root <none> and password file <none>
I20260806 00:49:06.942582 15950 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 00:49:06.942853 15950 webserver.cc:460] Webserver started at http://127.0.0.1:42105/ using document root <none> and password file <none>
2026-08-06T00:49:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:49:06Z Disabled control of system clock
I20260806 00:49:06.953325 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:41457
--webserver_interface=127.15.147.190
--webserver_port=0
--builtin_ntp_servers=127.15.147.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:41457
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44915/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 00:49:07.063628 23739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:07.063867 23739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:07.063903 23739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:07.067703 23739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:49:07.067752 23739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:07.067766 23739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:07.067781 23739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:49:07.067795 23739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 00:49:07.067808 23739 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 00:49:07.067821 23739 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44915/jwks.json
I20260806 00:49:07.072283 23739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:41457
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:41457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44915/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:07.073464 23739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:07.074870 23739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:07.083585 23747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:07.083606 23754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:07.083567 23746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:07.083838 23739 server_base.cc:1034] running on GCE node
I20260806 00:49:07.084467 23739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:07.085583 23739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:07.086748 23739 hybrid_clock.cc:648] HybridClock initialized: now 1785977347086729 us; error 30 us; skew 500 ppm
I20260806 00:49:07.088662 23739 webserver.cc:460] Webserver started at http://127.15.147.190:40269/ using document root <none> and password file <none>
I20260806 00:49:07.089238 23739 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:07.089330 23739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:07.089570 23739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:07.091714 23739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "5c4ac0b761534c81b4eebd9d9ff58620"
format_stamp: "Formatted at 2026-08-06 00:49:07 on dist-test-slave-45dx"
server_key: "c66acad3854140e267103fa4faca8b4a"
server_key_iv: "fbdc53b1afeb30f8ee263ed57272186b"
server_key_version: "encryptionkey@0"
I20260806 00:49:07.092350 23739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "5c4ac0b761534c81b4eebd9d9ff58620"
format_stamp: "Formatted at 2026-08-06 00:49:07 on dist-test-slave-45dx"
server_key: "c66acad3854140e267103fa4faca8b4a"
server_key_iv: "fbdc53b1afeb30f8ee263ed57272186b"
server_key_version: "encryptionkey@0"
I20260806 00:49:07.095963 23739 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 00:49:07.098412 23764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:07.099596 23739 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:49:07.099732 23739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "5c4ac0b761534c81b4eebd9d9ff58620"
format_stamp: "Formatted at 2026-08-06 00:49:07 on dist-test-slave-45dx"
server_key: "c66acad3854140e267103fa4faca8b4a"
server_key_iv: "fbdc53b1afeb30f8ee263ed57272186b"
server_key_version: "encryptionkey@0"
I20260806 00:49:07.099844 23739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:07.116130 23739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:07.116855 23739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:07.117050 23739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:07.124182 23739 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:41457
I20260806 00:49:07.124194 23835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:41457 every 8 connection(s)
I20260806 00:49:07.125146 23739 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:49:07.128252 23836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:07.129254 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 23739
I20260806 00:49:07.129387 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:49:07.129719 15950 external_mini_cluster.cc:1428] Setting key ec40e0f9af6b6ac84d3a158ed0e0a160
I20260806 00:49:07.134522 23836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620: Bootstrap starting.
I20260806 00:49:07.136816 23836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:07.137694 23836 log.cc:824] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:07.139267 23842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:07.131158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38912 (local address 127.15.147.190:41457)
0806 00:49:07.131828 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:07.131843 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:07.131902 (+ 59us) server_negotiation.cc:407] Connection header received
0806 00:49:07.132602 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:07.132621 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:07.133011 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:07.133342 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:07.133787 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:07.134802 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:07.135704 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:07.136183 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:07.136378 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:07.137293 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:07.137299 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:07.137313 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:07.137788 (+ 475us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:07.137871 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:07.137898 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:07.138002 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:07.138109 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 00:49:07.138384 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":124,"threads_started":1}
I20260806 00:49:07.139827 23836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620: No bootstrap required, opened a new log
I20260806 00:49:07.142071 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:49:07.143921 23836 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4ac0b761534c81b4eebd9d9ff58620" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41457 } }
I20260806 00:49:07.144125 23836 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:07.144199 23836 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c4ac0b761534c81b4eebd9d9ff58620, State: Initialized, Role: FOLLOWER
I20260806 00:49:07.144731 23836 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4ac0b761534c81b4eebd9d9ff58620" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41457 } }
I20260806 00:49:07.144945 23836 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:49:07.145053 23836 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:49:07.145175 23836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:07.146091 23836 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4ac0b761534c81b4eebd9d9ff58620" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41457 } }
I20260806 00:49:07.146451 23836 leader_election.cc:302] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c4ac0b761534c81b4eebd9d9ff58620; no voters:
I20260806 00:49:07.146746 23836 leader_election.cc:288] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:49:07.146867 23844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:07.147121 23844 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620 [term 1 LEADER]: Becoming Leader. State: Replica: 5c4ac0b761534c81b4eebd9d9ff58620, State: Running, Role: LEADER
I20260806 00:49:07.147404 23844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4ac0b761534c81b4eebd9d9ff58620" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41457 } }
I20260806 00:49:07.147966 23836 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:49:07.149253 23848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c4ac0b761534c81b4eebd9d9ff58620. Latest consensus state: current_term: 1 leader_uuid: "5c4ac0b761534c81b4eebd9d9ff58620" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4ac0b761534c81b4eebd9d9ff58620" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41457 } } }
I20260806 00:49:07.149394 23848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:07.149890 23859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:49:07.149253 23847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c4ac0b761534c81b4eebd9d9ff58620" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c4ac0b761534c81b4eebd9d9ff58620" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41457 } } }
I20260806 00:49:07.149992 23847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:07.152506 23859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:49:07.157343 23859 catalog_manager.cc:1347] Generated new cluster ID: bbaee7e7426e4cc2beb1ba243ecee853
I20260806 00:49:07.157406 23859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:49:07.164781 23859 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:49:07.165858 23859 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:49:07.191543 23859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c4ac0b761534c81b4eebd9d9ff58620: Generated new TSK 0
I20260806 00:49:07.192502 23859 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:49:07.208899 23842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:07.204010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38916 (local address 127.15.147.190:41457)
0806 00:49:07.204171 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:07.204175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:07.204394 (+ 219us) server_negotiation.cc:407] Connection header received
0806 00:49:07.204574 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:07.204577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:07.204624 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:07.204752 (+ 128us) server_negotiation.cc:226] Negotiated authn=JWT
0806 00:49:07.205012 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:07.205557 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:07.206445 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:07.206668 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:07.206773 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:07.207081 (+ 308us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 00:49:07.208420 (+ 1339us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44915/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 00:49:07.208761 (+ 341us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38916: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44915/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":79}
W20260806 00:49:07.209241 23842 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38916: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44915/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 00:49:07.210788 23878 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:07.203831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:41457 (local address 127.0.0.1:38916)
0806 00:49:07.204251 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:07.204263 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:07.204432 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:07.204755 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:07.204758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:07.204766 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 00:49:07.204897 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:07.204904 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:07.205679 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:07.205683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:07.206311 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:07.206319 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:07.206792 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:07.206796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:07.206888 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:07.206910 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 00:49:07.208586 (+ 1676us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44915/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 00:49:07.210621 (+ 2035us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.147.190:41457: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44915/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":1813,"thread_start_us":113,"threads_started":1}
I20260806 00:49:07.211598 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 23739
E20260806 00:49:07.224386 23879 webserver.cc:533] Webserver: sslize failed:
E20260806 00:49:07.224483 23879 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T00:49:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (317 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T00:49:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:49:07Z Disabled control of system clock
I20260806 00:49:07.269567 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:38449
--webserver_interface=127.15.147.190
--webserver_port=0
--builtin_ntp_servers=127.15.147.148:38635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:38449
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/testchainca.cert with env {}
W20260806 00:49:07.382068 23887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:07.382316 23887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:07.382354 23887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:07.386230 23887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:49:07.386281 23887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:07.386296 23887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:07.386312 23887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:49:07.386327 23887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 00:49:07.386341 23887 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 00:49:07.386354 23887 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 00:49:07.391412 23887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:38635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:38449
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:38449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:07.392355 23887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:07.393302 23887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:49:07.401827 23887 server_base.cc:1034] running on GCE node
W20260806 00:49:07.401902 23896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:07.401836 23899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:07.401865 23895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:07.402693 23887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:07.403720 23887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:07.404917 23887 hybrid_clock.cc:648] HybridClock initialized: now 1785977347404901 us; error 35 us; skew 500 ppm
I20260806 00:49:07.406772 23887 webserver.cc:460] Webserver started at http://127.15.147.190:41657/ using document root <none> and password file <none>
I20260806 00:49:07.407368 23887 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:07.407464 23887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:07.407691 23887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:07.409729 23887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "23ef1d62f2504d52883ac2b85294864b"
format_stamp: "Formatted at 2026-08-06 00:49:07 on dist-test-slave-45dx"
server_key: "fd99c219b6013518ae219be2edf5e135"
server_key_iv: "bc371e3116e5a29f2e9855a1467ce072"
server_key_version: "encryptionkey@0"
I20260806 00:49:07.410307 23887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "23ef1d62f2504d52883ac2b85294864b"
format_stamp: "Formatted at 2026-08-06 00:49:07 on dist-test-slave-45dx"
server_key: "fd99c219b6013518ae219be2edf5e135"
server_key_iv: "bc371e3116e5a29f2e9855a1467ce072"
server_key_version: "encryptionkey@0"
I20260806 00:49:07.413986 23887 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 00:49:07.416622 23905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:07.417908 23887 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:49:07.418056 23887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "23ef1d62f2504d52883ac2b85294864b"
format_stamp: "Formatted at 2026-08-06 00:49:07 on dist-test-slave-45dx"
server_key: "fd99c219b6013518ae219be2edf5e135"
server_key_iv: "bc371e3116e5a29f2e9855a1467ce072"
server_key_version: "encryptionkey@0"
I20260806 00:49:07.418177 23887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:07.435992 23887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:07.436738 23887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:07.436964 23887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:07.443929 23887 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:38449
I20260806 00:49:07.443938 23981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:38449 every 8 connection(s)
I20260806 00:49:07.444976 23887 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:49:07.445382 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 23887
I20260806 00:49:07.445528 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:49:07.445832 15950 external_mini_cluster.cc:1428] Setting key d7b3e8339c2b1f32840bb1c8c7dfcb1f
I20260806 00:49:07.448385 23982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:07.453948 23982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b: Bootstrap starting.
I20260806 00:49:07.454912 23985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:07.447216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46068 (local address 127.15.147.190:38449)
0806 00:49:07.447843 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:07.447857 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:07.447893 (+ 36us) server_negotiation.cc:407] Connection header received
0806 00:49:07.448620 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:07.448638 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:07.449031 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:07.449360 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:07.449795 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:07.450744 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:07.451632 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:07.452087 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:07.452260 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:07.453216 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:07.453221 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:07.453232 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:07.453624 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:07.453679 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:07.453698 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:07.453813 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:07.453970 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 00:49:07.454247 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":160,"threads_started":1}
I20260806 00:49:07.456458 23982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:07.457320 23982 log.cc:824] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:07.457727 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:49:07.459296 23982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b: No bootstrap required, opened a new log
I20260806 00:49:07.462826 23982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ef1d62f2504d52883ac2b85294864b" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 38449 } }
I20260806 00:49:07.463049 23982 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:07.463125 23982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23ef1d62f2504d52883ac2b85294864b, State: Initialized, Role: FOLLOWER
I20260806 00:49:07.463528 23982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ef1d62f2504d52883ac2b85294864b" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 38449 } }
I20260806 00:49:07.463665 23982 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:49:07.463753 23982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:49:07.463873 23982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:07.464813 23982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ef1d62f2504d52883ac2b85294864b" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 38449 } }
I20260806 00:49:07.465147 23982 leader_election.cc:302] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23ef1d62f2504d52883ac2b85294864b; no voters:
I20260806 00:49:07.465441 23982 leader_election.cc:288] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:49:07.465583 23988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:07.465818 23988 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b [term 1 LEADER]: Becoming Leader. State: Replica: 23ef1d62f2504d52883ac2b85294864b, State: Running, Role: LEADER
I20260806 00:49:07.466127 23988 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ef1d62f2504d52883ac2b85294864b" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 38449 } }
I20260806 00:49:07.466585 23982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:49:07.467847 23990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23ef1d62f2504d52883ac2b85294864b. Latest consensus state: current_term: 1 leader_uuid: "23ef1d62f2504d52883ac2b85294864b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ef1d62f2504d52883ac2b85294864b" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 38449 } } }
I20260806 00:49:07.467880 23989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23ef1d62f2504d52883ac2b85294864b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23ef1d62f2504d52883ac2b85294864b" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 38449 } } }
I20260806 00:49:07.468025 23990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:07.468029 23989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:07.468488 24002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:49:07.471596 24002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:49:07.477169 24002 catalog_manager.cc:1347] Generated new cluster ID: b181058675a34bd4911d3bc0ec6ca634
I20260806 00:49:07.477244 24002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:49:07.511299 24002 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:49:07.512235 24002 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:49:07.520331 24002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23ef1d62f2504d52883ac2b85294864b: Generated new TSK 0
I20260806 00:49:07.521018 24002 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:49:07.784178 23985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:07.530372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46084 (local address 127.15.147.190:38449)
0806 00:49:07.530526 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:07.530531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:07.530759 (+ 228us) server_negotiation.cc:407] Connection header received
0806 00:49:07.530952 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:07.530955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:07.531005 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:07.531093 (+ 88us) server_negotiation.cc:226] Negotiated authn=JWT
0806 00:49:07.531377 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:07.531896 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:07.533206 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:07.533429 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:07.533509 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:07.533891 (+ 382us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 00:49:07.783723 (+249832us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 00:49:07.784047 (+ 324us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46084: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 00:49:07.784314 23985 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46084: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 00:49:07.784785 24026 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:07.530312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:38449 (local address 127.0.0.1:46084)
0806 00:49:07.530659 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:07.530674 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:07.530796 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:07.531121 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:07.531125 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:07.531134 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 00:49:07.531257 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:07.531264 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:07.532035 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:07.532039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:07.533088 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:07.533100 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:07.533560 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:07.533564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:07.533666 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:07.533698 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 00:49:07.784005 (+250307us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 00:49:07.784190 (+ 185us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.147.190:38449: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":147,"threads_started":1}
I20260806 00:49:07.785439 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 23887
2026-08-06T00:49:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (554 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (128 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (130 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:49:08 dist-test-slave-45dx krb5kdc[24042](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:49:08 dist-test-slave-45dx krb5kdc[24042](info): set up 2 sockets
Aug 06 00:49:08 dist-test-slave-45dx krb5kdc[24042](info): commencing operation
krb5kdc: starting...
W20260806 00:49:09.257859 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:49:09 dist-test-slave-45dx krb5kdc[24042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:49:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:49:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:49:09.391603 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:39775
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:39775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:09.506359 24056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:09.506716 24056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:09.506801 24056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:09.510905 24056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:49:09.510998 24056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:09.511039 24056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:09.511097 24056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:49:09.511137 24056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:49:09.516578 24056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:39775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:39775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:09.517678 24056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:09.518718 24056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:09.527463 24067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:09.527519 24056 server_base.cc:1034] running on GCE node
W20260806 00:49:09.527537 24065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:09.527463 24070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:09.528427 24056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:09.529542 24056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:09.530762 24056 hybrid_clock.cc:648] HybridClock initialized: now 1785977349530727 us; error 44 us; skew 500 ppm
Aug 06 00:49:09 dist-test-slave-45dx krb5kdc[24042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977349, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:09.533648 24056 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:49:09.534569 24056 webserver.cc:460] Webserver started at http://127.15.147.190:37791/ using document root <none> and password file <none>
I20260806 00:49:09.535153 24056 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:09.535250 24056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:09.535501 24056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:09.537712 24056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "65f89ebdae414549aaee38f4479c6e51"
format_stamp: "Formatted at 2026-08-06 00:49:09 on dist-test-slave-45dx"
server_key: "fc14ba7863081ef5d6a302a6df4484af"
server_key_iv: "899072cdc649a5ea9443c9b5b50d9310"
server_key_version: "encryptionkey@0"
I20260806 00:49:09.538331 24056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "65f89ebdae414549aaee38f4479c6e51"
format_stamp: "Formatted at 2026-08-06 00:49:09 on dist-test-slave-45dx"
server_key: "fc14ba7863081ef5d6a302a6df4484af"
server_key_iv: "899072cdc649a5ea9443c9b5b50d9310"
server_key_version: "encryptionkey@0"
I20260806 00:49:09.542116 24056 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:49:09.544582 24080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:09.545835 24056 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:49:09.545979 24056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "65f89ebdae414549aaee38f4479c6e51"
format_stamp: "Formatted at 2026-08-06 00:49:09 on dist-test-slave-45dx"
server_key: "fc14ba7863081ef5d6a302a6df4484af"
server_key_iv: "899072cdc649a5ea9443c9b5b50d9310"
server_key_version: "encryptionkey@0"
I20260806 00:49:09.546090 24056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:09.566627 24056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:09.569885 24056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:09.570108 24056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:09.578027 24056 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:39775
I20260806 00:49:09.578058 24162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:39775 every 8 connection(s)
I20260806 00:49:09.579068 24056 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:49:09.581984 24163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:09.587702 24163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51: Bootstrap starting.
I20260806 00:49:09.589025 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 24056
I20260806 00:49:09.589162 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:49:09.589474 15950 external_mini_cluster.cc:1428] Setting key d63e9052492234dffc89288cf56eae85
I20260806 00:49:09.590688 24163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:09.591531 24163 log.cc:824] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:09.593442 24163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51: No bootstrap required, opened a new log
I20260806 00:49:09.597245 24163 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f89ebdae414549aaee38f4479c6e51" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39775 } }
I20260806 00:49:09.597512 24163 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:09.597597 24163 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65f89ebdae414549aaee38f4479c6e51, State: Initialized, Role: FOLLOWER
I20260806 00:49:09.598132 24163 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f89ebdae414549aaee38f4479c6e51" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39775 } }
I20260806 00:49:09.598311 24163 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:49:09.598389 24163 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 06 00:49:09 dist-test-slave-45dx krb5kdc[24042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:49:09.598495 24163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:09.599432 24163 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f89ebdae414549aaee38f4479c6e51" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39775 } }
I20260806 00:49:09.599778 24163 leader_election.cc:302] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65f89ebdae414549aaee38f4479c6e51; no voters:
I20260806 00:49:09.600147 24163 leader_election.cc:288] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:49:09.600260 24171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:09.600518 24171 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51 [term 1 LEADER]: Becoming Leader. State: Replica: 65f89ebdae414549aaee38f4479c6e51, State: Running, Role: LEADER
I20260806 00:49:09.600930 24171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f89ebdae414549aaee38f4479c6e51" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39775 } }
I20260806 00:49:09.601611 24163 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:49:09.602170 24171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65f89ebdae414549aaee38f4479c6e51. Latest consensus state: current_term: 1 leader_uuid: "65f89ebdae414549aaee38f4479c6e51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f89ebdae414549aaee38f4479c6e51" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39775 } } }
I20260806 00:49:09.602957 24171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:09.602830 24172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65f89ebdae414549aaee38f4479c6e51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65f89ebdae414549aaee38f4479c6e51" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39775 } } }
I20260806 00:49:09.603289 24172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:09.604312 24182 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:49:09.605212 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:09.590691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51566 (local address 127.15.147.190:39775)
0806 00:49:09.591170 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:09.591185 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:09.591221 (+ 36us) server_negotiation.cc:407] Connection header received
0806 00:49:09.591921 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:09.591940 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:09.592244 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:09.592627 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:09.593502 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:09.594512 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:09.595411 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:09.595895 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:09.596074 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:09.599057 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:09.599062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:09.599076 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:09.599103 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:09.601221 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:09.601751 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:09.601770 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:09.601776 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:09.601990 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:09.602374 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:09.602378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:09.602392 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:09.602969 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:09.603194 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:09.603934 (+ 740us) server_negotiation.cc:299] Negotiation successful
0806 00:49:09.604289 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":163,"threads_started":1}
I20260806 00:49:09.607731 24182 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:49:09.613648 24182 catalog_manager.cc:1347] Generated new cluster ID: 3ea031838dd74be691ee7e2aae4c8914
I20260806 00:49:09.613754 24182 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:49:09.644897 24182 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:49:09.646318 24182 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:49:09.653770 24182 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65f89ebdae414549aaee38f4479c6e51: Generated new TSK 0
I20260806 00:49:09.654659 24182 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
I20260806 00:49:09.687397 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39775
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:09.805177 24197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:09.805433 24197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:09.805509 24197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:09.809481 24197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:09.809536 24197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:09.809631 24197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:49:09.814682 24197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:09.815795 24197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:09.816928 24197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:09.827279 24210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:09.827279 24206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:09.827306 24205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:09.827559 24197 server_base.cc:1034] running on GCE node
I20260806 00:49:09.828070 24197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:09.828786 24197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:09.829973 24197 hybrid_clock.cc:648] HybridClock initialized: now 1785977349829945 us; error 45 us; skew 500 ppm
Aug 06 00:49:09 dist-test-slave-45dx krb5kdc[24042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977349, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:09.832752 24197 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:49:09.833621 24197 webserver.cc:460] Webserver started at http://127.15.147.129:39001/ using document root <none> and password file <none>
I20260806 00:49:09.834183 24197 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:09.834277 24197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:09.834517 24197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:09.836663 24197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "1b09f4eae36e44ecaeb2eef94a786e9f"
format_stamp: "Formatted at 2026-08-06 00:49:09 on dist-test-slave-45dx"
server_key: "e8dca7c81dfcb6fd623d5ba2e679d7bc"
server_key_iv: "003c75d70a4d55277e8e44c7103ba4e4"
server_key_version: "encryptionkey@0"
I20260806 00:49:09.837268 24197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "1b09f4eae36e44ecaeb2eef94a786e9f"
format_stamp: "Formatted at 2026-08-06 00:49:09 on dist-test-slave-45dx"
server_key: "e8dca7c81dfcb6fd623d5ba2e679d7bc"
server_key_iv: "003c75d70a4d55277e8e44c7103ba4e4"
server_key_version: "encryptionkey@0"
I20260806 00:49:09.841063 24197 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:49:09.843559 24220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:09.844823 24197 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:49:09.844977 24197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "1b09f4eae36e44ecaeb2eef94a786e9f"
format_stamp: "Formatted at 2026-08-06 00:49:09 on dist-test-slave-45dx"
server_key: "e8dca7c81dfcb6fd623d5ba2e679d7bc"
server_key_iv: "003c75d70a4d55277e8e44c7103ba4e4"
server_key_version: "encryptionkey@0"
I20260806 00:49:09.845096 24197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:09.866406 24197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:09.869351 24197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:09.869563 24197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:09.870173 24197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:09.871009 24197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:09.871084 24197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:09.871158 24197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:09.871201 24197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:09.880493 24197 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:35827
I20260806 00:49:09.880584 24387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:35827 every 8 connection(s)
I20260806 00:49:09.881378 24197 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:49:09.885555 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 24197
I20260806 00:49:09.885674 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:49:09.885998 15950 external_mini_cluster.cc:1428] Setting key c2f68de237d69cd748177188cc53fd96
Aug 06 00:49:09 dist-test-slave-45dx krb5kdc[24042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977349, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:49:09.896693 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:09.883169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:40645 (local address 127.15.147.190:39775)
0806 00:49:09.883337 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:09.883343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:09.884211 (+ 868us) server_negotiation.cc:407] Connection header received
0806 00:49:09.885598 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:09.885605 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:09.885670 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:09.885772 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:09.887189 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:09.887950 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:09.889155 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:09.889564 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:09.889690 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:09.892845 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:09.892848 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:09.892850 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:09.892872 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:09.894328 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:09.894999 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:09.895001 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:09.895002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:09.895078 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:09.895497 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:09.895500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:09.895501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:09.895668 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:09.895741 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:09.896433 (+ 692us) server_negotiation.cc:299] Negotiation successful
0806 00:49:09.896539 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 00:49:09.897892 24390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:09.883548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:39775 (local address 127.15.147.129:40645)
0806 00:49:09.884027 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:09.884096 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:09.885374 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:09.885944 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:09.885950 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:09.886384 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:09.887040 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:09.887059 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:09.888087 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:09.888090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:09.889023 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:09.889034 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:09.889745 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:09.889751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:09.889977 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:09.890602 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:09.890628 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:09.892680 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:09.894454 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:09.894463 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:09.894481 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:09.894887 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:09.895172 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:09.895178 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:09.895182 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:09.895396 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:09.895778 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:09.895787 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:09.896106 (+ 319us) client_negotiation.cc:769] Sending connection context
0806 00:49:09.896368 (+ 262us) client_negotiation.cc:240] Negotiation successful
0806 00:49:09.896739 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":125,"threads_started":1}
WARNING: no policy specified for kudu/127.15.147.130@KRBTEST.COM; defaulting to no policy
I20260806 00:49:09.899098 24388 heartbeater.cc:347] Connected to a master server at 127.15.147.190:39775
I20260806 00:49:09.899394 24388 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:09.899849 24388 heartbeater.cc:511] Master 127.15.147.190:39775 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.130@KRBTEST.COM" created.
I20260806 00:49:09.901501 24114 ts_manager.cc:194] Registered new tserver with Master: 1b09f4eae36e44ecaeb2eef94a786e9f (127.15.147.129:35827)
I20260806 00:49:09.902704 24114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:40645
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
I20260806 00:49:09.919205 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39775
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:10.039739 24393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:10.039991 24393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:10.040047 24393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:10.043933 24393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:10.043988 24393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:10.044039 24393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:49:10.049957 24393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:10.052408 24393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:10.053833 24393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:10.064533 24401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:10.064567 24400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:10.064567 24403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:10.064725 24393 server_base.cc:1034] running on GCE node
I20260806 00:49:10.065275 24393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:10.065896 24393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:10.067060 24393 hybrid_clock.cc:648] HybridClock initialized: now 1785977350067028 us; error 44 us; skew 500 ppm
Aug 06 00:49:10 dist-test-slave-45dx krb5kdc[24042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977350, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:10.069937 24393 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:49:10.070923 24393 webserver.cc:460] Webserver started at http://127.15.147.130:46127/ using document root <none> and password file <none>
I20260806 00:49:10.071543 24393 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:10.071648 24393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:10.071893 24393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:10.074059 24393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "be21717fa4d040bf97a8c4927c895740"
format_stamp: "Formatted at 2026-08-06 00:49:10 on dist-test-slave-45dx"
server_key: "63e28557bfdeb5eefd5928ef6d636d79"
server_key_iv: "e328d9b494dd6d0e81061e6d6175d354"
server_key_version: "encryptionkey@0"
I20260806 00:49:10.074637 24393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "be21717fa4d040bf97a8c4927c895740"
format_stamp: "Formatted at 2026-08-06 00:49:10 on dist-test-slave-45dx"
server_key: "63e28557bfdeb5eefd5928ef6d636d79"
server_key_iv: "e328d9b494dd6d0e81061e6d6175d354"
server_key_version: "encryptionkey@0"
I20260806 00:49:10.078593 24393 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 00:49:10.081115 24413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:10.082334 24393 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:49:10.082475 24393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "be21717fa4d040bf97a8c4927c895740"
format_stamp: "Formatted at 2026-08-06 00:49:10 on dist-test-slave-45dx"
server_key: "63e28557bfdeb5eefd5928ef6d636d79"
server_key_iv: "e328d9b494dd6d0e81061e6d6175d354"
server_key_version: "encryptionkey@0"
I20260806 00:49:10.082592 24393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:10.101486 24393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:10.105098 24393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:10.105312 24393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:10.105917 24393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:10.106762 24393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:10.106810 24393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:10.106878 24393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:10.106914 24393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:10.115855 24393 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:45387
I20260806 00:49:10.115963 24587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:45387 every 8 connection(s)
I20260806 00:49:10.116832 24393 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:49:10.117717 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 24393
I20260806 00:49:10.117828 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:49:10.118139 15950 external_mini_cluster.cc:1428] Setting key 49c8af7d95f49fc4d77302c547494753
Aug 06 00:49:10 dist-test-slave-45dx krb5kdc[24042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977350, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.147.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.131@KRBTEST.COM" created.
I20260806 00:49:10.133015 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:10.118865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:44583 (local address 127.15.147.190:39775)
0806 00:49:10.119015 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:10.119019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:10.119914 (+ 895us) server_negotiation.cc:407] Connection header received
0806 00:49:10.122002 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:10.122007 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:10.122074 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:10.122176 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:10.123318 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:10.123943 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:10.125805 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:10.126020 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:10.126103 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:10.129067 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:10.129071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:10.129073 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:10.129097 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:10.130654 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:10.131238 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:10.131246 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:10.131248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:10.131331 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:10.131692 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:10.131695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:10.131697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:10.131885 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:10.132005 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:10.132720 (+ 715us) server_negotiation.cc:299] Negotiation successful
0806 00:49:10.132854 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:49:10.133822 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:10.119245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:39775 (local address 127.15.147.130:44583)
0806 00:49:10.119734 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:10.119792 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:10.121735 (+ 1943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:10.122346 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:10.122352 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:10.122695 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:10.123169 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:10.123182 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:10.124083 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:10.124089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:10.125678 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:10.125693 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:10.126193 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:10.126200 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:10.126439 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:10.127085 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:10.127111 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:10.128893 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:10.130829 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:10.130835 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:10.130854 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:10.131111 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:10.131429 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:10.131433 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:10.131435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:10.131584 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:10.132006 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:10.132013 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:10.132430 (+ 417us) client_negotiation.cc:769] Sending connection context
0806 00:49:10.132657 (+ 227us) client_negotiation.cc:240] Negotiation successful
0806 00:49:10.133010 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":121,"threads_started":1}
I20260806 00:49:10.135074 24588 heartbeater.cc:347] Connected to a master server at 127.15.147.190:39775
I20260806 00:49:10.135383 24588 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:10.135989 24588 heartbeater.cc:511] Master 127.15.147.190:39775 requested a full tablet report, sending...
I20260806 00:49:10.137135 24114 ts_manager.cc:194] Registered new tserver with Master: be21717fa4d040bf97a8c4927c895740 (127.15.147.130:45387)
I20260806 00:49:10.138103 24114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:44583
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
I20260806 00:49:10.151324 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39775
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:10.266662 24597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:10.266961 24597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:10.267077 24597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:10.272605 24597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:10.272675 24597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:10.272761 24597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:49:10.278769 24597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:10.279872 24597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:10.280979 24597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:10.290853 24608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:10.290836 24604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:10.290935 24597 server_base.cc:1034] running on GCE node
W20260806 00:49:10.290824 24603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:10.291616 24597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:10.292158 24597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:10.293316 24597 hybrid_clock.cc:648] HybridClock initialized: now 1785977350293300 us; error 28 us; skew 500 ppm
Aug 06 00:49:10 dist-test-slave-45dx krb5kdc[24042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977350, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:10.298853 24597 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:49:10.299711 24597 webserver.cc:460] Webserver started at http://127.15.147.131:36641/ using document root <none> and password file <none>
I20260806 00:49:10.300251 24597 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:10.300385 24597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:10.300623 24597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:10.302739 24597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "3a40ce0bf6d14101b89b90bf103c43e5"
format_stamp: "Formatted at 2026-08-06 00:49:10 on dist-test-slave-45dx"
server_key: "1b1e02afc39ea021f44048b20aad1f0b"
server_key_iv: "c93fbccbb3a1fc2dd95c6d9df4bec356"
server_key_version: "encryptionkey@0"
I20260806 00:49:10.303308 24597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "3a40ce0bf6d14101b89b90bf103c43e5"
format_stamp: "Formatted at 2026-08-06 00:49:10 on dist-test-slave-45dx"
server_key: "1b1e02afc39ea021f44048b20aad1f0b"
server_key_iv: "c93fbccbb3a1fc2dd95c6d9df4bec356"
server_key_version: "encryptionkey@0"
I20260806 00:49:10.306900 24597 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 00:49:10.309209 24615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:10.310351 24597 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:49:10.310472 24597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "3a40ce0bf6d14101b89b90bf103c43e5"
format_stamp: "Formatted at 2026-08-06 00:49:10 on dist-test-slave-45dx"
server_key: "1b1e02afc39ea021f44048b20aad1f0b"
server_key_iv: "c93fbccbb3a1fc2dd95c6d9df4bec356"
server_key_version: "encryptionkey@0"
I20260806 00:49:10.310570 24597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:10.321791 24597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:10.324760 24597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:10.324966 24597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:10.325570 24597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:10.326413 24597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:10.326488 24597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:10.326553 24597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:10.326596 24597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:10.336004 24597 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:34167
I20260806 00:49:10.336050 24770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:34167 every 8 connection(s)
I20260806 00:49:10.337047 24597 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:49:10.338400 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 24597
I20260806 00:49:10.338526 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:49:10.338835 15950 external_mini_cluster.cc:1428] Setting key 31342885e9b48a0bde6a629820873521
Aug 06 00:49:10 dist-test-slave-45dx krb5kdc[24042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977350, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:49:10.352619 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:10.339081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:36973 (local address 127.15.147.190:39775)
0806 00:49:10.339270 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:10.339274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:10.340415 (+ 1141us) server_negotiation.cc:407] Connection header received
0806 00:49:10.342057 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:10.342060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:10.342111 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:10.342185 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:10.343618 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:10.344164 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:10.345429 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:10.345653 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:10.345736 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:10.348400 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:10.348405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:10.348413 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:10.348441 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:10.350128 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:10.350798 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:10.350801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:10.350802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:10.350911 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:10.351277 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:10.351282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:10.351284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:10.351525 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:10.351638 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:10.352285 (+ 647us) server_negotiation.cc:299] Negotiation successful
0806 00:49:10.352454 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 00:49:10.353758 24773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:10.339593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:39775 (local address 127.15.147.131:36973)
0806 00:49:10.340197 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:10.340261 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:10.341839 (+ 1578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:10.342358 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:10.342365 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:10.342759 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:10.343421 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:10.343438 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:10.344289 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:10.344292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:10.345306 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:10.345317 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:10.345805 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:10.345808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:10.345969 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:10.346587 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:10.346604 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:10.348167 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:10.350267 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:10.350274 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:10.350292 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:10.350676 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:10.350999 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:10.351003 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:10.351005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:10.351174 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:10.351639 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:10.351648 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:10.351987 (+ 339us) client_negotiation.cc:769] Sending connection context
0806 00:49:10.352244 (+ 257us) client_negotiation.cc:240] Negotiation successful
0806 00:49:10.352650 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":163,"threads_started":1}
I20260806 00:49:10.354889 24771 heartbeater.cc:347] Connected to a master server at 127.15.147.190:39775
I20260806 00:49:10.355158 24771 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:10.355696 24771 heartbeater.cc:511] Master 127.15.147.190:39775 requested a full tablet report, sending...
I20260806 00:49:10.356755 24114 ts_manager.cc:194] Registered new tserver with Master: 3a40ce0bf6d14101b89b90bf103c43e5 (127.15.147.131:34167)
I20260806 00:49:10.357331 24114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:36973
I20260806 00:49:10.365197 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/test-admin.keytab.
Aug 06 00:49:10 dist-test-slave-45dx krb5kdc[24042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:10.384908 15950 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 00:49:10.388072 24781 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:10.386373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57126 (local address 127.15.147.129:35827)
0806 00:49:10.386728 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:10.386748 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:10.386796 (+ 48us) server_negotiation.cc:407] Connection header received
0806 00:49:10.386878 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:10.386885 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:10.387087 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:10.387211 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:10.387869 (+ 658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57126: BlockingRecv error: recv got EOF from 127.0.0.1:57126 (error 108)
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":143,"threads_started":1}
W20260806 00:49:10.388101 15950 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/krb5cc' of type FILE
Aug 06 00:49:10 dist-test-slave-45dx krb5kdc[24042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:10.389210 15950 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 00:49:10 dist-test-slave-45dx krb5kdc[24042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:10.390142 15950 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 00:49:10 dist-test-slave-45dx krb5kdc[24042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.129@KRBTEST.COM
I20260806 00:49:10.400365 24781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:10.390993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57138 (local address 127.15.147.129:35827)
0806 00:49:10.391133 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:10.391137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:10.391232 (+ 95us) server_negotiation.cc:407] Connection header received
0806 00:49:10.391379 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:10.391384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:10.391435 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:10.391521 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:10.392051 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:10.392962 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:10.393790 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:10.394189 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:10.394307 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:10.396846 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:10.396856 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:10.396868 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:10.396891 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:10.398413 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:10.398936 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:10.398947 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:10.398949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:10.399047 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:10.399349 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:10.399353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:10.399355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:10.399601 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:10.399844 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:10.400009 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 00:49:10.400177 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 00:49:10.404449 24781 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:10.403395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57152 (local address 127.15.147.129:35827)
0806 00:49:10.403541 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:10.403544 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:10.403583 (+ 39us) server_negotiation.cc:407] Connection header received
0806 00:49:10.403743 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:10.403745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:10.403780 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:10.403845 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:10.404277 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57152: BlockingRecv error: recv got EOF from 127.0.0.1:57152 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 00:49:10.404518 15950 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/krb5cc' of type FILE
Aug 06 00:49:10 dist-test-slave-45dx krb5kdc[24042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:10.405515 15950 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 00:49:10 dist-test-slave-45dx krb5kdc[24042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:10.406394 15950 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 00:49:10 dist-test-slave-45dx krb5kdc[24042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.129@KRBTEST.COM
I20260806 00:49:10.414180 24781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:10.407133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57168 (local address 127.15.147.129:35827)
0806 00:49:10.407264 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:10.407268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:10.407283 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:49:10.407393 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:10.407395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:10.407428 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:10.407489 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:10.407926 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:10.408463 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:10.409183 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:10.409399 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:10.409470 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:10.411511 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:10.411514 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:10.411516 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:10.411536 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:10.412809 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:10.413179 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:10.413182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:10.413183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:10.413253 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:10.413475 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:10.413477 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:10.413478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:10.413621 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:10.413713 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:10.413924 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 00:49:10.414041 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 00:49:10.416714 24781 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:10.415783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57180 (local address 127.15.147.129:35827)
0806 00:49:10.415965 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:10.415968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:10.415998 (+ 30us) server_negotiation.cc:407] Connection header received
0806 00:49:10.416086 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:10.416088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:10.416119 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:10.416190 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:10.416606 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57180: BlockingRecv error: recv got EOF from 127.0.0.1:57180 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260806 00:49:10.417013 15950 init.cc:244] Error while opening credential cache '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785977317010501-15950-0/krb5kdc/krb5cc)
Aug 06 00:49:10 dist-test-slave-45dx krb5kdc[24042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:10.417927 15950 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 00:49:10 dist-test-slave-45dx krb5kdc[24042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:10.418730 15950 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 00:49:10 dist-test-slave-45dx krb5kdc[24042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.129@KRBTEST.COM
I20260806 00:49:10.426631 24781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:10.419427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57194 (local address 127.15.147.129:35827)
0806 00:49:10.419536 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:10.419542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:10.419639 (+ 97us) server_negotiation.cc:407] Connection header received
0806 00:49:10.419712 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:10.419714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:10.419744 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:10.419803 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:10.420237 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:10.420738 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:10.421430 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:10.421619 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:10.421690 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:10.423664 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:10.423667 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:10.423669 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:10.423690 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:10.425138 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:10.425531 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:10.425533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:10.425534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:10.425613 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:10.425846 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:10.425848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:10.425849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:10.425993 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:10.426090 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:10.426294 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 00:49:10.426456 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:49:10.427593 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 24197
I20260806 00:49:10.440600 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 24393
I20260806 00:49:10.453624 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 24597
I20260806 00:49:10.471226 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 24056
2026-08-06T00:49:10Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2423 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:49:10 dist-test-slave-45dx krb5kdc[24806](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:49:10 dist-test-slave-45dx krb5kdc[24806](info): set up 2 sockets
Aug 06 00:49:10 dist-test-slave-45dx krb5kdc[24806](info): commencing operation
krb5kdc: starting...
W20260806 00:49:11.689469 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:49:11 dist-test-slave-45dx krb5kdc[24806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:49:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:49:11Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.190.keytab.
Entry for principal oryx/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.190.keytab.
I20260806 00:49:11.819473 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:35655
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.190.keytab
--principal=oryx/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:36797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:35655
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:11.932140 24825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:11.932556 24825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:11.932670 24825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:11.936915 24825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:49:11.937005 24825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:11.937040 24825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:11.937079 24825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:49:11.937114 24825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:49:11.942493 24825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:36797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:35655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.190.keytab
--principal=oryx/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:35655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:11.943662 24825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:11.944818 24825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:11.953526 24834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:11.953532 24839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:11.953533 24836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:11.953583 24825 server_base.cc:1034] running on GCE node
I20260806 00:49:11.954443 24825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:11.955632 24825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:11.956796 24825 hybrid_clock.cc:648] HybridClock initialized: now 1785977351956776 us; error 33 us; skew 500 ppm
Aug 06 00:49:11 dist-test-slave-45dx krb5kdc[24806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977351, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:11.959547 24825 init.cc:377] Logged in from keytab as oryx/127.15.147.190@KRBTEST.COM (short username oryx)
I20260806 00:49:11.960546 24825 webserver.cc:460] Webserver started at http://127.15.147.190:45479/ using document root <none> and password file <none>
I20260806 00:49:11.961110 24825 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:11.961205 24825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:11.961444 24825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:11.963548 24825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "13ed091816be4886ac2042f64901402c"
format_stamp: "Formatted at 2026-08-06 00:49:11 on dist-test-slave-45dx"
server_key: "cd8187435a38009791446f2f8a74f07b"
server_key_iv: "fff163b2041f939548d83be8da673ebd"
server_key_version: "encryptionkey@0"
I20260806 00:49:11.964133 24825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "13ed091816be4886ac2042f64901402c"
format_stamp: "Formatted at 2026-08-06 00:49:11 on dist-test-slave-45dx"
server_key: "cd8187435a38009791446f2f8a74f07b"
server_key_iv: "fff163b2041f939548d83be8da673ebd"
server_key_version: "encryptionkey@0"
I20260806 00:49:11.967782 24825 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 00:49:11.970185 24852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:11.971411 24825 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:49:11.971557 24825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "13ed091816be4886ac2042f64901402c"
format_stamp: "Formatted at 2026-08-06 00:49:11 on dist-test-slave-45dx"
server_key: "cd8187435a38009791446f2f8a74f07b"
server_key_iv: "fff163b2041f939548d83be8da673ebd"
server_key_version: "encryptionkey@0"
I20260806 00:49:11.971673 24825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:12.001498 24825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:12.004683 24825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:12.004913 24825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:12.013098 24825 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:35655
I20260806 00:49:12.013402 24916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:35655 every 8 connection(s)
I20260806 00:49:12.014166 24825 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:49:12.017217 24920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:12.017581 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 24825
I20260806 00:49:12.017716 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:49:12.018071 15950 external_mini_cluster.cc:1428] Setting key e7abad6970122abdbb6e4505a05eda51
I20260806 00:49:12.024415 24920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c: Bootstrap starting.
Aug 06 00:49:12 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM
I20260806 00:49:12.027244 24920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:12.028093 24920 log.cc:824] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:12.030206 24920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c: No bootstrap required, opened a new log
I20260806 00:49:12.032641 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:12.019518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45818 (local address 127.15.147.190:35655)
0806 00:49:12.019923 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:12.019939 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:12.019974 (+ 35us) server_negotiation.cc:407] Connection header received
0806 00:49:12.020732 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:12.020750 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:12.021069 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:12.021331 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:12.022311 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:12.023261 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:12.024405 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:12.024927 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:12.025049 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:12.027811 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:12.027817 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:12.027831 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:12.027864 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:12.029737 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:12.030230 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:12.030235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:12.030239 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:12.030409 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:12.030752 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:12.030754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:12.030755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:12.031148 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:12.031288 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:12.031588 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 00:49:12.031898 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":126,"threads_started":1}
I20260806 00:49:12.034032 24920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } }
I20260806 00:49:12.034262 24920 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:12.034359 24920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ed091816be4886ac2042f64901402c, State: Initialized, Role: FOLLOWER
I20260806 00:49:12.034817 24920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } }
I20260806 00:49:12.034940 24920 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:49:12.034986 24920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:49:12.035051 24920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:12.035845 24920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } }
I20260806 00:49:12.036116 24920 leader_election.cc:302] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13ed091816be4886ac2042f64901402c; no voters:
I20260806 00:49:12.036423 24920 leader_election.cc:288] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:49:12.036576 24931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:12.036816 24931 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [term 1 LEADER]: Becoming Leader. State: Replica: 13ed091816be4886ac2042f64901402c, State: Running, Role: LEADER
I20260806 00:49:12.037168 24931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } }
I20260806 00:49:12.037564 24920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:49:12.039161 24934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13ed091816be4886ac2042f64901402c. Latest consensus state: current_term: 1 leader_uuid: "13ed091816be4886ac2042f64901402c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } }
I20260806 00:49:12.039292 24934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:12.039640 24933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13ed091816be4886ac2042f64901402c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } }
I20260806 00:49:12.040177 24933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: This master's current role is: LEADER
W20260806 00:49:12.042291 24954 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:49:12.042382 24954 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:49:12.042490 24952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:49:12.043397 24952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:49:12.048727 24952 catalog_manager.cc:1347] Generated new cluster ID: 6177584354ee43e8bdd6609d65673d28
I20260806 00:49:12.048797 24952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:49:12.060030 24952 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:49:12.061430 24952 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:49:12.076974 24952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c: Generated new TSK 0
I20260806 00:49:12.077873 24952 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.129.keytab.
Entry for principal oryx/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.129.keytab.
I20260806 00:49:12.111703 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:35655
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.129.keytab
--principal=oryx/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:36797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:12.229507 24963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:12.229795 24963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:12.229847 24963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:12.234035 24963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:12.234112 24963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:12.234197 24963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:49:12.240237 24963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:36797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.129.keytab
--principal=oryx/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:35655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:12.241482 24963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:12.242615 24963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:12.252862 24974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:12.252825 24970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:12.252815 24963 server_base.cc:1034] running on GCE node
W20260806 00:49:12.252755 24969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:12.253510 24963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:12.254070 24963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:12.255259 24963 hybrid_clock.cc:648] HybridClock initialized: now 1785977352255221 us; error 45 us; skew 500 ppm
Aug 06 00:49:12 dist-test-slave-45dx krb5kdc[24806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977352, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:12.258097 24963 init.cc:377] Logged in from keytab as oryx/127.15.147.129@KRBTEST.COM (short username oryx)
I20260806 00:49:12.258944 24963 webserver.cc:460] Webserver started at http://127.15.147.129:34231/ using document root <none> and password file <none>
I20260806 00:49:12.259490 24963 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:12.259546 24963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:12.259719 24963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:12.261857 24963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad"
format_stamp: "Formatted at 2026-08-06 00:49:12 on dist-test-slave-45dx"
server_key: "0b8649932cd520a314cb97eed5371643"
server_key_iv: "895961716551032035356923ede9b67b"
server_key_version: "encryptionkey@0"
I20260806 00:49:12.262583 24963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad"
format_stamp: "Formatted at 2026-08-06 00:49:12 on dist-test-slave-45dx"
server_key: "0b8649932cd520a314cb97eed5371643"
server_key_iv: "895961716551032035356923ede9b67b"
server_key_version: "encryptionkey@0"
I20260806 00:49:12.273241 24963 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.006s sys 0.000s
I20260806 00:49:12.277266 24983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:12.278645 24963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:49:12.278810 24963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad"
format_stamp: "Formatted at 2026-08-06 00:49:12 on dist-test-slave-45dx"
server_key: "0b8649932cd520a314cb97eed5371643"
server_key_iv: "895961716551032035356923ede9b67b"
server_key_version: "encryptionkey@0"
I20260806 00:49:12.278955 24963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:12.291344 24963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:12.294301 24963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:12.294512 24963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:12.295269 24963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:12.296502 24963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:12.296571 24963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:12.296654 24963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:12.296682 24963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:12.306170 24963 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:35801
I20260806 00:49:12.306247 25155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:35801 every 8 connection(s)
I20260806 00:49:12.307121 24963 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:49:12.312309 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 24963
I20260806 00:49:12.312467 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:49:12.312820 15950 external_mini_cluster.cc:1428] Setting key 21ac63b906ff0a893ee1bdc4ff1d3c69
Aug 06 00:49:12 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977352, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM
I20260806 00:49:12.323724 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:12.309075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:59643 (local address 127.15.147.190:35655)
0806 00:49:12.309221 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:12.309225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:12.310078 (+ 853us) server_negotiation.cc:407] Connection header received
0806 00:49:12.311550 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:12.311554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:12.311617 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:12.311706 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:12.313477 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:12.314318 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:12.315595 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:12.315874 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:12.315998 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:12.319065 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:12.319073 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:12.319079 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:12.319119 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:12.321100 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:12.321673 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:12.321681 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:12.321686 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:12.321847 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:12.322226 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:12.322233 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:12.322236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:12.322564 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:12.322699 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:12.323324 (+ 625us) server_negotiation.cc:299] Negotiation successful
0806 00:49:12.323505 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":54}
I20260806 00:49:12.324621 25160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:12.309447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.129:59643)
0806 00:49:12.309899 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:12.309945 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:12.311323 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:12.311914 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:12.311921 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:12.312577 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:12.313313 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:12.313332 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:12.314460 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:12.314465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:12.315449 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:12.315460 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:12.316045 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:12.316050 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:12.316232 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:12.317015 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:12.317045 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:12.318875 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:12.321233 (+ 2358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:12.321239 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:12.321252 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:12.321527 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:12.321963 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:12.321965 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:12.321967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:12.322115 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:12.322729 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:12.322735 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:12.323012 (+ 277us) client_negotiation.cc:769] Sending connection context
0806 00:49:12.323222 (+ 210us) client_negotiation.cc:240] Negotiation successful
0806 00:49:12.323530 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for oryx/127.15.147.130@KRBTEST.COM; defaulting to no policy
I20260806 00:49:12.326089 25156 heartbeater.cc:347] Connected to a master server at 127.15.147.190:35655
I20260806 00:49:12.326352 25156 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:12.326879 25156 heartbeater.cc:511] Master 127.15.147.190:35655 requested a full tablet report, sending...
I20260806 00:49:12.328600 24871 ts_manager.cc:194] Registered new tserver with Master: e9c22b36dc1d43afbe4f6d3e2d2592ad (127.15.147.129:35801)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.147.130@KRBTEST.COM" created.
I20260806 00:49:12.329809 24871 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:59643
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.130.keytab.
Entry for principal oryx/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.130.keytab.
I20260806 00:49:12.347515 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:35655
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.130.keytab
--principal=oryx/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:36797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:12.466229 25164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:12.466560 25164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:12.466639 25164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:12.470857 25164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:12.470927 25164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:12.471036 25164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:49:12.476804 25164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:36797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.130.keytab
--principal=oryx/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:35655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:12.477954 25164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:12.479060 25164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:12.489271 25171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:12.489291 25174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:12.489291 25170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:12.489387 25164 server_base.cc:1034] running on GCE node
I20260806 00:49:12.490027 25164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:12.490679 25164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:12.491855 25164 hybrid_clock.cc:648] HybridClock initialized: now 1785977352491821 us; error 52 us; skew 500 ppm
Aug 06 00:49:12 dist-test-slave-45dx krb5kdc[24806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977352, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:12.494939 25164 init.cc:377] Logged in from keytab as oryx/127.15.147.130@KRBTEST.COM (short username oryx)
I20260806 00:49:12.496039 25164 webserver.cc:460] Webserver started at http://127.15.147.130:37489/ using document root <none> and password file <none>
I20260806 00:49:12.496737 25164 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:12.496807 25164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:12.497076 25164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:12.499352 25164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c"
format_stamp: "Formatted at 2026-08-06 00:49:12 on dist-test-slave-45dx"
server_key: "4f646fe4ab25674d8f99a86eae585b70"
server_key_iv: "08dffab1eccfa20794dc7c73490f5294"
server_key_version: "encryptionkey@0"
I20260806 00:49:12.500015 25164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c"
format_stamp: "Formatted at 2026-08-06 00:49:12 on dist-test-slave-45dx"
server_key: "4f646fe4ab25674d8f99a86eae585b70"
server_key_iv: "08dffab1eccfa20794dc7c73490f5294"
server_key_version: "encryptionkey@0"
I20260806 00:49:12.504066 25164 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:49:12.506672 25183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:12.508018 25164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:49:12.508150 25164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c"
format_stamp: "Formatted at 2026-08-06 00:49:12 on dist-test-slave-45dx"
server_key: "4f646fe4ab25674d8f99a86eae585b70"
server_key_iv: "08dffab1eccfa20794dc7c73490f5294"
server_key_version: "encryptionkey@0"
I20260806 00:49:12.508267 25164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:12.531322 25164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:12.535005 25164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:12.535238 25164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:12.535990 25164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:12.536950 25164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:12.537009 25164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:12.537083 25164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:12.537122 25164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:12.546804 25164 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:45675
I20260806 00:49:12.546852 25346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:45675 every 8 connection(s)
I20260806 00:49:12.547684 25164 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:49:12.556063 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 25164
I20260806 00:49:12.556183 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:49:12.556531 15950 external_mini_cluster.cc:1428] Setting key 654e45ce810f4d67a5b382448472715a
Aug 06 00:49:12 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977352, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM
I20260806 00:49:12.562130 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:12.549455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:50201 (local address 127.15.147.190:35655)
0806 00:49:12.549600 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:12.549603 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:12.550443 (+ 840us) server_negotiation.cc:407] Connection header received
0806 00:49:12.551831 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:12.551833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:12.551879 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:12.551952 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:12.553143 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:12.553794 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:12.554924 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:12.555153 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:12.555244 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:12.558228 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:12.558231 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:12.558233 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:12.558255 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:12.559776 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:12.560467 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:12.560473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:12.560475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:12.560578 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:12.560930 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:12.560934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:12.560937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:12.561204 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:12.561315 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:12.561845 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 00:49:12.561985 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:49:12.563236 25351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:12.549816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.130:50201)
0806 00:49:12.550255 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:12.550312 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:12.551628 (+ 1316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:12.552110 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:12.552116 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:12.552543 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:12.553013 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:12.553026 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:12.553930 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:12.553933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:12.554814 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:12.554823 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:12.555332 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:12.555336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:12.555503 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:12.556215 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:12.556237 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:12.558063 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:12.559913 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:12.559922 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:12.559940 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:12.560361 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:12.560679 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:12.560681 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:12.560682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:12.560820 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:12.561332 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:12.561337 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:12.561574 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 00:49:12.561724 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 00:49:12.562121 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":128,"threads_started":1}
I20260806 00:49:12.564778 25347 heartbeater.cc:347] Connected to a master server at 127.15.147.190:35655
I20260806 00:49:12.565073 25347 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:12.565587 25347 heartbeater.cc:511] Master 127.15.147.190:35655 requested a full tablet report, sending...
I20260806 00:49:12.566681 24871 ts_manager.cc:194] Registered new tserver with Master: 308ca6c0a0b7403ba8c7187f1fd4f98c (127.15.147.130:45675)
I20260806 00:49:12.567262 24871 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:50201
WARNING: no policy specified for oryx/127.15.147.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.147.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.131.keytab.
Entry for principal oryx/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.131.keytab.
I20260806 00:49:12.587574 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:35655
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.131.keytab
--principal=oryx/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:36797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:12.707129 25354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:12.707405 25354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:12.707449 25354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:12.711539 25354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:12.711608 25354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:12.711692 25354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:49:12.717144 25354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:36797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.131.keytab
--principal=oryx/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:35655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:12.718279 25354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:12.719420 25354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:12.730223 25363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:12.730255 25361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:12.730270 25360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:12.730465 25354 server_base.cc:1034] running on GCE node
I20260806 00:49:12.730912 25354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:12.731514 25354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:12.732681 25354 hybrid_clock.cc:648] HybridClock initialized: now 1785977352732663 us; error 26 us; skew 500 ppm
Aug 06 00:49:12 dist-test-slave-45dx krb5kdc[24806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977352, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:12.735631 25354 init.cc:377] Logged in from keytab as oryx/127.15.147.131@KRBTEST.COM (short username oryx)
I20260806 00:49:12.736726 25354 webserver.cc:460] Webserver started at http://127.15.147.131:45799/ using document root <none> and password file <none>
I20260806 00:49:12.737342 25354 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:12.737443 25354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:12.737720 25354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:12.740039 25354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "c84469497d4f45d6860a4c4603d326d0"
format_stamp: "Formatted at 2026-08-06 00:49:12 on dist-test-slave-45dx"
server_key: "92236056a89fb422180efc6dc2c58e04"
server_key_iv: "73401d8350428bc3ee3e83f70a5cb56f"
server_key_version: "encryptionkey@0"
I20260806 00:49:12.740720 25354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "c84469497d4f45d6860a4c4603d326d0"
format_stamp: "Formatted at 2026-08-06 00:49:12 on dist-test-slave-45dx"
server_key: "92236056a89fb422180efc6dc2c58e04"
server_key_iv: "73401d8350428bc3ee3e83f70a5cb56f"
server_key_version: "encryptionkey@0"
I20260806 00:49:12.744714 25354 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:49:12.747387 25374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:12.748756 25354 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:49:12.748915 25354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "c84469497d4f45d6860a4c4603d326d0"
format_stamp: "Formatted at 2026-08-06 00:49:12 on dist-test-slave-45dx"
server_key: "92236056a89fb422180efc6dc2c58e04"
server_key_iv: "73401d8350428bc3ee3e83f70a5cb56f"
server_key_version: "encryptionkey@0"
I20260806 00:49:12.749037 25354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:12.768138 25354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:12.771487 25354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:12.771739 25354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:12.772469 25354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:12.773433 25354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:12.773510 25354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:12.773610 25354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:12.773666 25354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:12.784394 25531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:36609 every 8 connection(s)
I20260806 00:49:12.784406 25354 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:36609
I20260806 00:49:12.785508 25354 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:49:12.795410 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 25354
I20260806 00:49:12.795536 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
Aug 06 00:49:12 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977352, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM
I20260806 00:49:12.795884 15950 external_mini_cluster.cc:1428] Setting key b8094a7c82b59e083224d647e8efa42e
I20260806 00:49:12.800012 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:12.787431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:59799 (local address 127.15.147.190:35655)
0806 00:49:12.787584 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:12.787588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:12.788583 (+ 995us) server_negotiation.cc:407] Connection header received
0806 00:49:12.790013 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:12.790017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:12.790083 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:12.790194 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:12.791407 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:12.791936 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:12.793113 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:12.793335 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:12.793444 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:12.796405 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:12.796412 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:12.796418 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:12.796451 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:12.798000 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:12.798564 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:12.798568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:12.798570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:12.798676 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:12.799049 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:12.799052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:12.799053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:12.799226 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:12.799301 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:12.799799 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 00:49:12.799892 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:49:12.800938 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:12.787812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.131:59799)
0806 00:49:12.788416 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:12.788463 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:12.789787 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:12.790373 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:12.790379 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:12.790752 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:12.791243 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:12.791256 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:12.792076 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:12.792086 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:12.792963 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:12.792974 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:12.793503 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:12.793507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:12.793672 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:12.794332 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:12.794348 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:12.796179 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:12.798098 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:12.798103 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:12.798118 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:12.798427 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:12.798796 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:12.798798 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:12.798799 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:12.798946 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:12.799324 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:12.799329 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:12.799593 (+ 264us) client_negotiation.cc:769] Sending connection context
0806 00:49:12.799749 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 00:49:12.800030 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":225,"threads_started":1}
I20260806 00:49:12.802129 25532 heartbeater.cc:347] Connected to a master server at 127.15.147.190:35655
I20260806 00:49:12.802343 25532 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:12.802775 25532 heartbeater.cc:511] Master 127.15.147.190:35655 requested a full tablet report, sending...
I20260806 00:49:12.803740 24871 ts_manager.cc:194] Registered new tserver with Master: c84469497d4f45d6860a4c4603d326d0 (127.15.147.131:36609)
I20260806 00:49:12.804411 24871 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:59799
I20260806 00:49:12.809710 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.147.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.147.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.147.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.189.keytab.
Entry for principal oryx/127.15.147.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.189.keytab.
I20260806 00:49:12.839214 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.189:33281
--webserver_interface=127.15.147.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.189.keytab
--principal=oryx/127.15.147.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:36797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:35655,127.15.147.189:33281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:12.953966 25538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:12.954250 25538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:12.954321 25538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:12.958272 25538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:49:12.958351 25538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:12.958389 25538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:12.958446 25538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:49:12.958485 25538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:49:12.963548 25538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:36797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:35655,127.15.147.189:33281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.189.keytab
--principal=oryx/127.15.147.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.189:33281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.147.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
W20260806 00:49:12.963783 25538 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.15.147.190:35655,127.15.147.189:33281'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 00:49:12.964655 25538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:12.965642 25538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:12.974087 25546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:12.974115 25543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:12.974253 25538 server_base.cc:1034] running on GCE node
W20260806 00:49:12.974123 25549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:12.975047 25538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:12.976269 25538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:12.977488 25538 hybrid_clock.cc:648] HybridClock initialized: now 1785977352977477 us; error 30 us; skew 500 ppm
Aug 06 00:49:12 dist-test-slave-45dx krb5kdc[24806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977352, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:12.981395 25538 init.cc:377] Logged in from keytab as oryx/127.15.147.189@KRBTEST.COM (short username oryx)
I20260806 00:49:12.983314 25538 webserver.cc:460] Webserver started at http://127.15.147.189:35123/ using document root <none> and password file <none>
I20260806 00:49:12.984386 25538 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:12.984519 25538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:12.984803 25538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:12.987138 25538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/data/instance:
uuid: "2a526db9ea2b4fdcad2f43cd537a87cd"
format_stamp: "Formatted at 2026-08-06 00:49:12 on dist-test-slave-45dx"
server_key: "38642d153789ae4c47a4856ceea7e87c"
server_key_iv: "7658285a8b9b89ef020ce76240e749ef"
server_key_version: "encryptionkey@0"
I20260806 00:49:12.987777 25538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/wal/instance:
uuid: "2a526db9ea2b4fdcad2f43cd537a87cd"
format_stamp: "Formatted at 2026-08-06 00:49:12 on dist-test-slave-45dx"
server_key: "38642d153789ae4c47a4856ceea7e87c"
server_key_iv: "7658285a8b9b89ef020ce76240e749ef"
server_key_version: "encryptionkey@0"
I20260806 00:49:12.991879 25538 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 00:49:12.994563 25558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:12.995816 25538 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:49:12.995965 25538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/wal
uuid: "2a526db9ea2b4fdcad2f43cd537a87cd"
format_stamp: "Formatted at 2026-08-06 00:49:12 on dist-test-slave-45dx"
server_key: "38642d153789ae4c47a4856ceea7e87c"
server_key_iv: "7658285a8b9b89ef020ce76240e749ef"
server_key_version: "encryptionkey@0"
I20260806 00:49:12.996088 25538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:13.018740 25538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:13.022744 25538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:13.022976 25538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:13.031231 25538 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.189:33281
I20260806 00:49:13.031549 25634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.189:33281 every 8 connection(s)
I20260806 00:49:13.032229 25538 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/data/info.pb
I20260806 00:49:13.034987 25635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:13.036890 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 25538
I20260806 00:49:13.039529 25635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.147.189@KRBTEST.COM
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977352, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.189@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM
I20260806 00:49:13.052058 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.041789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33798 (local address 127.15.147.190:35655)
0806 00:49:13.041918 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.041922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.042404 (+ 482us) server_negotiation.cc:407] Connection header received
0806 00:49:13.042578 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.042581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.042634 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.042724 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.043537 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.044102 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.045261 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.045563 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.045648 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.048412 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.048415 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.048416 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.048439 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.050123 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.050601 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.050605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.050606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.050688 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.050972 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.050976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.050978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.051197 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.051311 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.051746 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.051889 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:49:13.052219 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.041697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.0.0.1:33798)
0806 00:49:13.042184 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.042226 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.042470 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.042775 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.042781 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.043229 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.043397 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.043405 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.044230 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.044233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.045128 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.045138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.045750 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.045764 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.045897 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.046438 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.046459 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.048206 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.050235 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.050245 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.050251 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.050486 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.050784 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.050787 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.050789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.050878 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.051438 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.051443 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.051545 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.051687 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.051890 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":238,"threads_started":1}
I20260806 00:49:13.052266 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.038592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37042 (local address 127.15.147.189:33281)
0806 00:49:13.039010 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.039026 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.039061 (+ 35us) server_negotiation.cc:407] Connection header received
0806 00:49:13.039741 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.039759 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.040128 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.040529 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.041428 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.042955 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.043959 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.044615 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.044745 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.047804 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.047811 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.047825 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.047850 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.049465 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.049913 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.049919 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.049923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.050026 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.050309 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.050312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.050314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.050685 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.050938 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.051153 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.051423 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":135,"threads_started":1}
I20260806 00:49:13.054680 25635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977352, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.189@KRBTEST.COM for oryx/127.15.147.189@KRBTEST.COM
I20260806 00:49:13.062660 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.055168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:33281 (local address 127.0.0.1:37056)
0806 00:49:13.055407 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.055418 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.055479 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.055693 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.055695 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.055829 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.055917 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.055922 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.057130 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.057134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.057624 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.057632 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.058075 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.058078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.058148 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.058643 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.058656 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.059825 (+ 1169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.061326 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.061329 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.061330 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.061524 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.061832 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.061834 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.061835 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.061908 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.062236 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.062238 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.062313 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.062392 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.062487 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180}
I20260806 00:49:13.062674 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.055306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37056 (local address 127.15.147.189:33281)
0806 00:49:13.055438 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.055441 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.055455 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:49:13.055559 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.055561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.055595 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.055665 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.056012 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.056936 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.057722 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.057973 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.058052 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.059948 (+ 1896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.059951 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.059953 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.059973 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.061228 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.061645 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.061648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.061649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.061728 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.061987 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.061989 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.061990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.062154 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.062226 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.062465 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.062554 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 00:49:13.065701 25635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: Bootstrap starting.
I20260806 00:49:13.068017 25635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:13.068799 25635 log.cc:824] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:13.070714 25635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: No bootstrap required, opened a new log
I20260806 00:49:13.073824 25635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } }
I20260806 00:49:13.074028 25635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:13.074061 25635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a526db9ea2b4fdcad2f43cd537a87cd, State: Initialized, Role: FOLLOWER
I20260806 00:49:13.074484 25635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } }
I20260806 00:49:13.075011 25642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } } }
I20260806 00:49:13.075196 25642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:49:13.075736 25635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [sys.catalog]: configured and running, proceeding with master startup.
W20260806 00:49:13.080574 25657 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:49:13.080655 25657 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:49:13.083205 25538 master_runner.cc:428] Detected that this master 2a526db9ea2b4fdcad2f43cd537a87cd is joining an existing cluster
I20260806 00:49:13.083256 25538 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.147.189:33281
I20260806 00:49:13.093611 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.085300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33810 (local address 127.15.147.190:35655)
0806 00:49:13.085429 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.085435 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.085638 (+ 203us) server_negotiation.cc:407] Connection header received
0806 00:49:13.085820 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.085824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.085876 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.085960 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.086498 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.087259 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.088434 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.088743 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.088858 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.089928 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.089932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.089934 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.089965 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.091826 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.092337 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.092346 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.092348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.092425 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.092758 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.092762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.092764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.092987 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.093090 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.093333 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.093448 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 00:49:13.093622 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.085224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.0.0.1:33810)
0806 00:49:13.085529 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.085546 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.085660 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.085973 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.085975 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.086221 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.086364 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.086370 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.087399 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.087402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.088279 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.088289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.088872 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.088875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.088959 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.089461 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.089472 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.089786 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.091948 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.091952 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.091957 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.092228 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.092523 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.092527 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.092529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.092653 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.093086 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.093089 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.093161 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.093258 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.093395 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":123,"threads_started":1}
I20260806 00:49:13.093768 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.085299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37068 (local address 127.15.147.189:33281)
0806 00:49:13.085421 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.085426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.085878 (+ 452us) server_negotiation.cc:407] Connection header received
0806 00:49:13.086019 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.086022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.086080 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.086128 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.086764 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.087436 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.088162 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.088486 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.088586 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.090440 (+ 1854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.090444 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.090446 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.090476 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.092126 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.092643 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.092647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.092648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.092720 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.092971 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.092974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.092975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.093174 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.093267 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.093522 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.093615 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":42}
I20260806 00:49:13.094110 25665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.085490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:33281 (local address 127.0.0.1:37068)
0806 00:49:13.085794 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.085807 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.085917 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.086259 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.086262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.086507 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.086645 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.086653 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.087538 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.087541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.088054 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.088060 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.088623 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.088627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.088721 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.089876 (+ 1155us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.089891 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.090296 (+ 405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.092238 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.092242 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.092244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.092534 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.092793 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.092797 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.092799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.092875 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.093298 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.093301 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.093374 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.093644 (+ 270us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.094003 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":115,"threads_started":1}
I20260806 00:49:13.099956 25642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:49:13.100111 25642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } }
I20260806 00:49:13.100960 25642 leader_election.cc:288] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ed091816be4886ac2042f64901402c (127.15.147.190:35655)
I20260806 00:49:13.101347 24884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ed091816be4886ac2042f64901402c" is_pre_election: true
I20260806 00:49:13.101922 25559 leader_election.cc:302] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 2a526db9ea2b4fdcad2f43cd537a87cd; no voters: 13ed091816be4886ac2042f64901402c
I20260806 00:49:13.102196 25642 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977351, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.190@KRBTEST.COM for oryx/127.15.147.189@KRBTEST.COM
I20260806 00:49:13.109072 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.098614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37074 (local address 127.15.147.189:33281)
0806 00:49:13.098731 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.098735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.099050 (+ 315us) server_negotiation.cc:407] Connection header received
0806 00:49:13.099255 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.099258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.099310 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.099412 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.100069 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.100903 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.102511 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.102705 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.102800 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.105293 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.105296 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.105298 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.105328 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.106969 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.107555 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.107559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.107561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.107649 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.108036 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.108042 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.108045 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.108281 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.108396 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.108782 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.108893 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 00:49:13.109081 25666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.098581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:33281 (local address 127.0.0.1:37074)
0806 00:49:13.098935 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.098976 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.099141 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.099461 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.099467 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.099807 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.099945 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.099954 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.101115 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.101119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.102380 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.102393 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.102810 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.102813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.102920 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.103431 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.103445 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.105133 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.107092 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.107105 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.107114 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.107446 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.107749 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.107751 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.107753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.107902 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.108484 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.108489 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.108585 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.108728 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.108911 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":90,"threads_started":1}
I20260806 00:49:13.110575 24871 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.147.189:33281: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd"
member_type: NON_VOTER
last_known_addr {
host: "127.15.147.189"
port: 33281
}
attrs {
promote: true
}
}
dest_uuid: "13ed091816be4886ac2042f64901402c"
cas_config_opid_index: -1
I20260806 00:49:13.111227 24871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: NON_VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: true } }
I20260806 00:49:13.113183 25672 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2a526db9ea2b4fdcad2f43cd537a87cd (127.15.147.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: NON_VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: true } } }
I20260806 00:49:13.114090 25600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:13.114600 25672 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.147.189:33281
I20260806 00:49:13.114840 25672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "13ed091816be4886ac2042f64901402c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: NON_VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: true } } }
I20260806 00:49:13.114948 25672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:13.115121 25538 master.cc:537] Master@127.15.147.189:33281 shutting down...
I20260806 00:49:13.115486 25600 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 1 FOLLOWER]: Refusing update from remote peer 13ed091816be4886ac2042f64901402c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 00:49:13.116143 25671 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: NON_VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 00:49:13.119395 24857 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 00:49:13.120012 25538 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:13.120349 25538 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:13.120491 25538 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: stopping tablet replica
W20260806 00:49:13.120491 24857 consensus_peers.cc:487] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c -> Peer 2a526db9ea2b4fdcad2f43cd537a87cd (127.15.147.189:33281): Couldn't send request to peer 2a526db9ea2b4fdcad2f43cd537a87cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
WARNING: no policy specified for oryx/127.15.147.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.147.188@KRBTEST.COM" created.
I20260806 00:49:13.125486 25538 master.cc:552] Master@127.15.147.189:33281 shutdown complete.
I20260806 00:49:13.125556 25538 master_runner.cc:305] Clearing existing system tablet
I20260806 00:49:13.125792 25538 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:13.125861 25538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:13.127242 25538 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 00:49:13.128301 25680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:13.128582 25538 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:13.128698 25538 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 00:49:13.128729 25538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/wal
uuid: "2a526db9ea2b4fdcad2f43cd537a87cd"
format_stamp: "Formatted at 2026-08-06 00:49:12 on dist-test-slave-45dx"
server_key: "38642d153789ae4c47a4856ceea7e87c"
server_key_iv: "7658285a8b9b89ef020ce76240e749ef"
server_key_version: "encryptionkey@0"
I20260806 00:49:13.130944 25538 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:13.133464 25538 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 00:49:13.133572 25538 log.cc:1196] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 00:49:13.133828 25538 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: Deleting consensus metadata
I20260806 00:49:13.133999 25538 master_runner.cc:315] Copying system tablet from 127.15.147.190:35655
I20260806 00:49:13.135612 25538 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: tablet copy: Beginning tablet copy session from remote peer at address 127.15.147.190:35655
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.147.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.188.keytab.
Entry for principal oryx/127.15.147.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.188.keytab.
I20260806 00:49:13.143572 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.188:34207
--webserver_interface=127.15.147.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.188.keytab
--principal=oryx/127.15.147.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:36797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:35655,127.15.147.189:33281,127.15.147.188:34207
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:49:13.145449 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.136708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33826 (local address 127.15.147.190:35655)
0806 00:49:13.136856 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.136859 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.136873 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:49:13.136917 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.136919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.136962 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.137034 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.137647 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.138168 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.139342 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.139664 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.139747 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.141234 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.141237 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.141239 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.141265 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.143251 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.143805 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.143808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.143810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.143902 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.144224 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.144231 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.144236 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.144575 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.144690 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.145056 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.145205 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:49:13.145516 25690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.136223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.0.0.1:33826)
0806 00:49:13.136550 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.136566 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.136683 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.137083 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.137086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.137364 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.137488 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.137504 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.138278 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.138281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.139226 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.139238 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.139781 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.139785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.139884 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.140570 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.140586 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.141031 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.143393 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.143396 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.143399 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.143678 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.144018 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.144021 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.144023 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.144113 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.144726 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.144729 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.144846 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.144974 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.145134 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":124,"threads_started":1}
I20260806 00:49:13.146522 24896 tablet_copy_service.cc:140] P 13ed091816be4886ac2042f64901402c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2a526db9ea2b4fdcad2f43cd537a87cd ({username='oryx', principal='oryx/127.15.147.189@KRBTEST.COM'} at 127.0.0.1:33826)
I20260806 00:49:13.146692 24896 tablet_copy_service.cc:161] P 13ed091816be4886ac2042f64901402c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2a526db9ea2b4fdcad2f43cd537a87cd at {username='oryx', principal='oryx/127.15.147.189@KRBTEST.COM'} at 127.0.0.1:33826: session id = 2a526db9ea2b4fdcad2f43cd537a87cd-00000000000000000000000000000000
I20260806 00:49:13.148722 24896 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 00:49:13.150892 25538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 00:49:13.154381 25538 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: tablet copy: Starting download of 0 data blocks...
I20260806 00:49:13.154632 25538 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: tablet copy: Starting download of 1 WAL segments...
I20260806 00:49:13.156791 25538 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 00:49:13.158746 24896 tablet_copy_service.cc:342] P 13ed091816be4886ac2042f64901402c: Request end of tablet copy session 2a526db9ea2b4fdcad2f43cd537a87cd-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.147.189@KRBTEST.COM'} at 127.0.0.1:33826
I20260806 00:49:13.158856 24896 tablet_copy_service.cc:433] P 13ed091816be4886ac2042f64901402c: ending tablet copy session 2a526db9ea2b4fdcad2f43cd537a87cd-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2a526db9ea2b4fdcad2f43cd537a87cd
I20260806 00:49:13.167096 25538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:13.168949 25693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:13.169596 25698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:13.169723 25694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:13.175146 25538 server_base.cc:1034] running on GCE node
I20260806 00:49:13.175454 25538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:13.175791 25538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:13.176946 25538 hybrid_clock.cc:648] HybridClock initialized: now 1785977353176930 us; error 32 us; skew 500 ppm
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977353, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:13.178669 25538 init.cc:377] Logged in from keytab as oryx/127.15.147.189@KRBTEST.COM (short username oryx)
I20260806 00:49:13.179145 25538 webserver.cc:460] Webserver started at http://127.15.147.189:34965/ using document root <none> and password file <none>
I20260806 00:49:13.179340 25538 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:13.179383 25538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:13.180809 25538 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 00:49:13.181924 25705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:13.182206 25538 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 00:49:13.182286 25538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/wal
uuid: "2a526db9ea2b4fdcad2f43cd537a87cd"
format_stamp: "Formatted at 2026-08-06 00:49:12 on dist-test-slave-45dx"
server_key: "38642d153789ae4c47a4856ceea7e87c"
server_key_iv: "7658285a8b9b89ef020ce76240e749ef"
server_key_version: "encryptionkey@0"
I20260806 00:49:13.182374 25538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:13.205407 25538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:13.207564 25538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:13.212656 25538 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.189:33281
I20260806 00:49:13.213073 25772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.189:33281 every 8 connection(s)
I20260806 00:49:13.213346 25538 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-1/data/info.pb
I20260806 00:49:13.215266 25774 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:49:13.216410 25774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: Bootstrap starting.
I20260806 00:49:13.227763 25774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:49:13.228130 25774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: Bootstrap complete.
I20260806 00:49:13.228822 25774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: NON_VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: true } }
I20260806 00:49:13.228968 25774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2a526db9ea2b4fdcad2f43cd537a87cd, State: Initialized, Role: LEARNER
I20260806 00:49:13.229074 25774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: NON_VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: true } }
I20260806 00:49:13.229521 25777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: NON_VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: true } } }
I20260806 00:49:13.229635 25774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:49:13.229725 25777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [sys.catalog]: This master's current role is: LEARNER
I20260806 00:49:13.233021 25790 catalog_manager.cc:1259] Loaded cluster ID: 6177584354ee43e8bdd6609d65673d28
I20260806 00:49:13.233150 25790 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: loading cluster ID for follower catalog manager: success
I20260806 00:49:13.235363 25790 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: acquiring CA information for follower catalog manager: success
I20260806 00:49:13.236830 25790 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 00:49:13.277521 25691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:13.277889 25691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:13.278012 25691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:13.282195 25691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:49:13.282289 25691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:13.282325 25691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:13.282361 25691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:49:13.282392 25691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:49:13.288445 25691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:36797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:35655,127.15.147.189:33281,127.15.147.188:34207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.188.keytab
--principal=oryx/127.15.147.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.188:34207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.147.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:13.289662 25691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:13.290738 25691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:49:13.299618 25691 server_base.cc:1034] running on GCE node
W20260806 00:49:13.299576 25798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:13.299576 25797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:13.299597 25800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:13.300622 25691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:13.301790 25691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:13.302963 25691 hybrid_clock.cc:648] HybridClock initialized: now 1785977353302950 us; error 27 us; skew 500 ppm
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977353, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:13.305815 25691 init.cc:377] Logged in from keytab as oryx/127.15.147.188@KRBTEST.COM (short username oryx)
I20260806 00:49:13.306734 25691 webserver.cc:460] Webserver started at http://127.15.147.188:34429/ using document root <none> and password file <none>
I20260806 00:49:13.307284 25691 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:13.307387 25691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:13.307614 25691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:13.309684 25691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/data/instance:
uuid: "4358f4e37da5459594e616128989e93a"
format_stamp: "Formatted at 2026-08-06 00:49:13 on dist-test-slave-45dx"
server_key: "554e44c711aa5d6efecd0067a16e7d7c"
server_key_iv: "b1fa873028a053c067d1dc697abb1d37"
server_key_version: "encryptionkey@0"
I20260806 00:49:13.310258 25691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/wal/instance:
uuid: "4358f4e37da5459594e616128989e93a"
format_stamp: "Formatted at 2026-08-06 00:49:13 on dist-test-slave-45dx"
server_key: "554e44c711aa5d6efecd0067a16e7d7c"
server_key_iv: "b1fa873028a053c067d1dc697abb1d37"
server_key_version: "encryptionkey@0"
I20260806 00:49:13.314451 25691 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 00:49:13.317335 25811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:13.318629 25691 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:49:13.318811 25691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/wal
uuid: "4358f4e37da5459594e616128989e93a"
format_stamp: "Formatted at 2026-08-06 00:49:13 on dist-test-slave-45dx"
server_key: "554e44c711aa5d6efecd0067a16e7d7c"
server_key_iv: "b1fa873028a053c067d1dc697abb1d37"
server_key_version: "encryptionkey@0"
I20260806 00:49:13.318969 25691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:13.332151 25156 heartbeater.cc:503] Master 127.15.147.190:35655 was elected leader, sending a full tablet report...
I20260806 00:49:13.335628 25691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:13.338825 25691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:13.339047 25691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:13.346567 25691 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.188:34207
I20260806 00:49:13.346903 25888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.188:34207 every 8 connection(s)
I20260806 00:49:13.347591 25691 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/data/info.pb
I20260806 00:49:13.350406 25890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:13.352578 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 25691
I20260806 00:49:13.354730 25890 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.147.188@KRBTEST.COM
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977353, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.188@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM
I20260806 00:49:13.367549 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.357279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33832 (local address 127.15.147.190:35655)
0806 00:49:13.357431 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.357435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.357987 (+ 552us) server_negotiation.cc:407] Connection header received
0806 00:49:13.358160 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.358163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.358216 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.358303 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.358975 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.359496 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.360420 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.360734 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.360881 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.363508 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.363514 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.363520 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.363557 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.365102 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.366050 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.366054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.366056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.366143 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.366440 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.366443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.366445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.366629 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.366739 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.367250 (+ 511us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.367380 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:49:13.368181 25893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.354190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53864 (local address 127.15.147.188:34207)
0806 00:49:13.354600 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.354622 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.354670 (+ 48us) server_negotiation.cc:407] Connection header received
0806 00:49:13.355478 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.355499 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.355896 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.356181 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.357151 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.358495 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.359382 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.359849 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.360012 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.363011 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.363020 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.363037 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.363060 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.365163 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.365571 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.365576 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.365581 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.365696 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.366000 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.366002 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.366003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.366421 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.366639 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.366926 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.367261 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":268,"thread_start_us":138,"threads_started":1}
I20260806 00:49:13.369343 25895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.357081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.0.0.1:33832)
0806 00:49:13.357856 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.357897 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.358037 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.358353 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.358359 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.358721 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.358841 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.358847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.359625 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.359628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.360254 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.360263 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.360908 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.360912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.360996 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.361535 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.361552 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.363358 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.365572 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.365580 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.365585 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.365923 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.366242 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.366246 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.366248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.366340 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.366739 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.366747 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.366891 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.367043 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.369198 (+ 2155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":61,"thread_start_us":311,"threads_started":1}
I20260806 00:49:13.370569 25890 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977353, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.188@KRBTEST.COM for oryx/127.15.147.189@KRBTEST.COM
I20260806 00:49:13.378782 25895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.371076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:33281 (local address 127.0.0.1:37084)
0806 00:49:13.371196 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.371212 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.371287 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.371663 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.371665 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.371792 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.371885 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.371890 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.372850 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.372852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.373372 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.373379 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.373834 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.373837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.373938 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.374488 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.374499 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.375788 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.377459 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.377462 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.377464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.377675 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.377948 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.377950 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.377951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.378030 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.378372 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.378374 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.378455 (+ 81us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.378548 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.378656 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 00:49:13.378902 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.371205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37084 (local address 127.15.147.189:33281)
0806 00:49:13.371454 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.371458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.371470 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:49:13.371514 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.371517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.371556 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.371636 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.371988 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.372752 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.373472 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.373697 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.373775 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.375941 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.375944 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.375946 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.375971 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.377350 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.377763 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.377766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.377767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.377845 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.378109 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.378112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.378113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.378291 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.378370 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.378606 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.378712 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":106,"threads_started":1}
I20260806 00:49:13.379678 25890 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977353, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.188@KRBTEST.COM for oryx/127.15.147.188@KRBTEST.COM
I20260806 00:49:13.387422 25895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.380206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34207 (local address 127.0.0.1:53872)
0806 00:49:13.380345 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.380366 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.380428 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.380613 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.380615 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.380741 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.380812 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.380816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.381494 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.381502 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.382087 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.382096 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.382550 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.382552 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.382629 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.383126 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.383138 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.384494 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.386155 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.386157 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.386158 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.386335 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.386607 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.386609 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.386610 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.386683 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.387000 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.387002 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.387072 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.387149 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.387251 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 00:49:13.387428 25893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.380298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53872 (local address 127.15.147.188:34207)
0806 00:49:13.380440 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.380443 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.380456 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:49:13.380485 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.380487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.380519 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.380584 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.380907 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.381403 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.382187 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.382416 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.382545 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.384618 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.384621 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.384623 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.384644 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.386062 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.386428 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.386431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.386432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.386522 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.386760 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.386762 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.386763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.386920 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.386995 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.387181 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.387303 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":62}
I20260806 00:49:13.391010 25890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: Bootstrap starting.
I20260806 00:49:13.393389 25890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:13.394124 25890 log.cc:824] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:13.396423 25890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: No bootstrap required, opened a new log
I20260806 00:49:13.399462 25890 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } }
I20260806 00:49:13.399708 25890 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:13.399798 25890 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4358f4e37da5459594e616128989e93a, State: Initialized, Role: FOLLOWER
I20260806 00:49:13.400292 25890 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } }
I20260806 00:49:13.400893 25902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } } }
I20260806 00:49:13.401026 25902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:49:13.401669 25890 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [sys.catalog]: configured and running, proceeding with master startup.
W20260806 00:49:13.406813 25919 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:49:13.406903 25919 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:49:13.411043 25691 master_runner.cc:428] Detected that this master 4358f4e37da5459594e616128989e93a is joining an existing cluster
I20260806 00:49:13.411105 25691 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.147.188:34207
I20260806 00:49:13.422789 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 24963
I20260806 00:49:13.432512 25893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.415033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53878 (local address 127.15.147.188:34207)
0806 00:49:13.415138 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.415142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.415410 (+ 268us) server_negotiation.cc:407] Connection header received
0806 00:49:13.415580 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.415584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.415623 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.415726 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.417240 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.418043 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.419839 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.420279 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.420388 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.422184 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.422189 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.422193 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.422228 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.429620 (+ 7392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.430487 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.430494 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.430499 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.430633 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.431080 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.431087 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.431091 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.431405 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.431498 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.432043 (+ 545us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.432208 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:49:13.433372 25930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.414740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34207 (local address 127.0.0.1:53878)
0806 00:49:13.415327 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.415342 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.415465 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.415753 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.415757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.416032 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.417113 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.417122 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.418176 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.418182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.419693 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.419712 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.420475 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.420478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.420631 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.421344 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.421360 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.422019 (+ 659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.429979 (+ 7960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.429986 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.429991 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.430355 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.430807 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.430813 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.430818 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.430958 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.431585 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.431591 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.431725 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.432690 (+ 965us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.432894 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":894,"spinlock_wait_cycles":8448,"thread_start_us":92,"threads_started":1}
I20260806 00:49:13.443194 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.414075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33848 (local address 127.15.147.190:35655)
0806 00:49:13.414229 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.414236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.414656 (+ 420us) server_negotiation.cc:407] Connection header received
0806 00:49:13.414950 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.414954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.415018 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.415082 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.416186 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.416849 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.418244 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.418672 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.418760 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.421434 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.421438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.421440 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.421471 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.440239 (+ 18768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.440856 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.440862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.440866 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.441279 (+ 413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.441966 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.441973 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.441978 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.442292 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.442378 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.442798 (+ 420us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.442948 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:49:13.443920 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.414024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.0.0.1:33848)
0806 00:49:13.414559 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.414579 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.414741 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.415258 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.415261 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.415916 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.416063 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.416072 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.416990 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.416995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.418095 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.418105 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.418947 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.418951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.419052 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.420333 (+ 1281us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.420358 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.421266 (+ 908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.440439 (+ 19173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.440445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.440450 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.440728 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.441412 (+ 684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.441418 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.441423 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.441836 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.442474 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.442480 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.442599 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.443360 (+ 761us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.443559 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":22,"thread_start_us":116,"threads_started":1}
I20260806 00:49:13.445626 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.414076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37092 (local address 127.15.147.189:33281)
0806 00:49:13.414313 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.414318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.414922 (+ 604us) server_negotiation.cc:407] Connection header received
0806 00:49:13.415111 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.415114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.415166 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.415216 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.416553 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.417492 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.419073 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.419499 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.419595 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.440604 (+ 21009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.440609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.440612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.440647 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.442612 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.443209 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.443213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.443215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.443326 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.444492 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.444499 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.444503 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.444772 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.444869 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.445214 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.445372 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260806 00:49:13.446128 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:35801
--local_ip_for_outbound_sockets=127.15.147.129
--tserver_master_addrs=127.15.147.190:35655
--webserver_port=34231
--webserver_interface=127.15.147.129
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.129.keytab
--principal=oryx/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:36797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.147.190:35655,127.15.147.189:33281
--tserver_master_addrs=127.15.147.190:35655,127.15.147.189:33281,127.15.147.188:34207 with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:49:13.446238 25929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.414081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:33281 (local address 127.0.0.1:37092)
0806 00:49:13.414704 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.414719 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.414997 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.415438 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.415441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.416021 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.416397 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.416408 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.417643 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.417648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.418907 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.418919 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.419761 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.419764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.419993 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.439813 (+ 19820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.439830 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.440394 (+ 564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.442764 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.442768 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.442770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.443074 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.443815 (+ 741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.443819 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.443821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.443984 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.444917 (+ 933us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.444920 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.445020 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.445784 (+ 764us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.445952 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":2,"spinlock_wait_cycles":33792,"thread_start_us":363,"threads_started":1}
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977351, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.190@KRBTEST.COM for oryx/127.15.147.188@KRBTEST.COM
I20260806 00:49:13.457541 25666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.449581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34207 (local address 127.0.0.1:53880)
0806 00:49:13.449725 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.449742 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.449848 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.450120 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.450123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.450319 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.450407 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.450412 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.451142 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.451147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.451800 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.451810 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.452253 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.452257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.452453 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.452956 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.452976 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.454281 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.456004 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.456007 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.456009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.456214 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.456541 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.456544 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.456546 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.456636 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.457009 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.457012 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.457115 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.457220 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.457376 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 00:49:13.457541 25893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.449628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53880 (local address 127.15.147.188:34207)
0806 00:49:13.449730 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.449734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.449793 (+ 59us) server_negotiation.cc:407] Connection header received
0806 00:49:13.449965 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.449967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.450008 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.450103 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.450539 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.451033 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.451922 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.452142 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.452250 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.454416 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.454420 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.454422 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.454454 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.455863 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.456347 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.456350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.456352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.456442 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.456733 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.456737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.456739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.456909 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.457016 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.457274 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.457380 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 00:49:13.459123 24871 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.147.188:34207: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "4358f4e37da5459594e616128989e93a"
member_type: NON_VOTER
last_known_addr {
host: "127.15.147.188"
port: 34207
}
attrs {
promote: true
}
}
dest_uuid: "13ed091816be4886ac2042f64901402c"
cas_config_opid_index: 5
I20260806 00:49:13.459707 24871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: NON_VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: true } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: NON_VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: true } }
I20260806 00:49:13.460866 25671 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4358f4e37da5459594e616128989e93a (127.15.147.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: NON_VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: true } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: NON_VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: true } } }
I20260806 00:49:13.462203 25671 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.147.188:34207
I20260806 00:49:13.462394 25671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "13ed091816be4886ac2042f64901402c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: NON_VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: true } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: NON_VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: true } } }
I20260806 00:49:13.462508 25671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:13.462966 25691 master.cc:537] Master@127.15.147.188:34207 shutting down...
W20260806 00:49:13.464901 24857 consensus_peers.cc:487] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c -> Peer 4358f4e37da5459594e616128989e93a (127.15.147.188:34207): Couldn't send request to peer 4358f4e37da5459594e616128989e93a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 00:49:13.470247 25666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.463041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:33281 (local address 127.0.0.1:37098)
0806 00:49:13.465551 (+ 2510us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.465569 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.465680 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.466099 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.466105 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.466128 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:13.466238 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.466246 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.467291 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.467294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.468402 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.468414 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.469823 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.469826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.469919 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.469970 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.470044 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.470114 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2429}
I20260806 00:49:13.470750 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.463485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37098 (local address 127.15.147.189:33281)
0806 00:49:13.463627 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.463634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.465642 (+ 2008us) server_negotiation.cc:407] Connection header received
0806 00:49:13.465771 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.465776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.465843 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.465935 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:13.466376 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.467161 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.468579 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.469398 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.469569 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.469748 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.470522 (+ 774us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.470616 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:49:13.471531 25736 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 1 LEARNER]: Refusing update from remote peer 13ed091816be4886ac2042f64901402c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 00:49:13.471536 25691 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:13.472030 25672 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: NON_VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 00:49:13.472128 25691 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:13.472242 25691 tablet_replica.cc:330] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: stopping tablet replica
I20260806 00:49:13.473985 25736 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4358f4e37da5459594e616128989e93a (127.15.147.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: NON_VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: true } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: NON_VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: true } } }
I20260806 00:49:13.476862 25777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13ed091816be4886ac2042f64901402c. Latest consensus state: current_term: 1 leader_uuid: "13ed091816be4886ac2042f64901402c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: NON_VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: true } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: NON_VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: true } } }
I20260806 00:49:13.477000 25777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [sys.catalog]: This master's current role is: LEARNER
I20260806 00:49:13.477106 25777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "13ed091816be4886ac2042f64901402c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: NON_VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: true } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: NON_VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: true } } }
I20260806 00:49:13.477200 25777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [sys.catalog]: This master's current role is: LEARNER
I20260806 00:49:13.483719 25672 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c: attempting to promote NON_VOTER 2a526db9ea2b4fdcad2f43cd537a87cd to VOTER
I20260806 00:49:13.484148 25672 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: NON_VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: true } }
I20260806 00:49:13.485641 25666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.485228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34207 (local address 127.0.0.1:53888)
0806 00:49:13.485477 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.485559 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174}
I20260806 00:49:13.485700 25736 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 1 LEARNER]: Refusing update from remote peer 13ed091816be4886ac2042f64901402c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 00:49:13.486130 25934 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 00:49:13.486631 25666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.486427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34207 (local address 127.0.0.1:53902)
0806 00:49:13.486516 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.486567 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 00:49:13.486805 24857 consensus_peers.cc:487] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c -> Peer 4358f4e37da5459594e616128989e93a (127.15.147.188:34207): Couldn't send request to peer 4358f4e37da5459594e616128989e93a. Status: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34207: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 00:49:13.488013 25671 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2a526db9ea2b4fdcad2f43cd537a87cd (127.15.147.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: NON_VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: true } } }
I20260806 00:49:13.489679 25691 master.cc:552] Master@127.15.147.188:34207 shutdown complete.
I20260806 00:49:13.489727 25691 master_runner.cc:305] Clearing existing system tablet
I20260806 00:49:13.489812 25671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "13ed091816be4886ac2042f64901402c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: NON_VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: true } } }
I20260806 00:49:13.489915 25671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:13.490003 25691 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:13.490060 25691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:13.490053 25671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13ed091816be4886ac2042f64901402c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: NON_VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: true } } }
I20260806 00:49:13.490137 25671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:13.490892 25736 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2a526db9ea2b4fdcad2f43cd537a87cd (127.15.147.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: NON_VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: true } } }
I20260806 00:49:13.492245 25933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "13ed091816be4886ac2042f64901402c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: NON_VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: true } } }
I20260806 00:49:13.492396 25933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:49:13.495901 25691 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.000s
I20260806 00:49:13.496954 25936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:13.497184 25691 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:13.497329 25691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 00:49:13.497366 25691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/wal
uuid: "4358f4e37da5459594e616128989e93a"
format_stamp: "Formatted at 2026-08-06 00:49:13 on dist-test-slave-45dx"
server_key: "554e44c711aa5d6efecd0067a16e7d7c"
server_key_iv: "b1fa873028a053c067d1dc697abb1d37"
server_key_version: "encryptionkey@0"
I20260806 00:49:13.499938 25691 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:13.502393 25691 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 00:49:13.502518 25691 log.cc:1196] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 00:49:13.502900 25691 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: Deleting consensus metadata
I20260806 00:49:13.503077 25691 master_runner.cc:315] Copying system tablet from 127.15.147.190:35655
I20260806 00:49:13.504288 25691 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: tablet copy: Beginning tablet copy session from remote peer at address 127.15.147.190:35655
I20260806 00:49:13.512926 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.505166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33854 (local address 127.15.147.190:35655)
0806 00:49:13.505304 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.505309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.505325 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:49:13.505363 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.505366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.505413 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.505501 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.506145 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.506680 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.507782 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.508035 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.508146 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.509506 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.509509 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.509512 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.509536 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.511052 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.511523 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.511526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.511528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.511608 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.511873 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.511876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.511878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.512058 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.512163 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.512657 (+ 494us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.512766 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:49:13.512943 25944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.504832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.0.0.1:33854)
0806 00:49:13.505096 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.505111 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.505231 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.505526 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.505529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.505747 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.506019 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.506027 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.506788 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.506793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.507646 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.507656 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.508150 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.508153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.508329 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.508910 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.508926 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.509343 (+ 417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.511146 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.511150 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.511153 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.511417 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.511698 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.511701 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.511703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.511787 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.512226 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.512231 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.512365 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.512598 (+ 233us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.512753 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":102,"threads_started":1}
I20260806 00:49:13.513671 24896 tablet_copy_service.cc:140] P 13ed091816be4886ac2042f64901402c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 4358f4e37da5459594e616128989e93a ({username='oryx', principal='oryx/127.15.147.188@KRBTEST.COM'} at 127.0.0.1:33854)
I20260806 00:49:13.513788 24896 tablet_copy_service.cc:161] P 13ed091816be4886ac2042f64901402c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 4358f4e37da5459594e616128989e93a at {username='oryx', principal='oryx/127.15.147.188@KRBTEST.COM'} at 127.0.0.1:33854: session id = 4358f4e37da5459594e616128989e93a-00000000000000000000000000000000
I20260806 00:49:13.514636 24896 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 00:49:13.515796 25691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 00:49:13.517735 25691 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: tablet copy: Starting download of 0 data blocks...
I20260806 00:49:13.517942 25691 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: tablet copy: Starting download of 1 WAL segments...
I20260806 00:49:13.519236 25691 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 00:49:13.521001 24896 tablet_copy_service.cc:342] P 13ed091816be4886ac2042f64901402c: Request end of tablet copy session 4358f4e37da5459594e616128989e93a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.147.188@KRBTEST.COM'} at 127.0.0.1:33854
I20260806 00:49:13.521121 24896 tablet_copy_service.cc:433] P 13ed091816be4886ac2042f64901402c: ending tablet copy session 4358f4e37da5459594e616128989e93a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 4358f4e37da5459594e616128989e93a
I20260806 00:49:13.529036 25691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:13.530854 25945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:13.530885 25948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:13.531387 25691 server_base.cc:1034] running on GCE node
W20260806 00:49:13.532451 25946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:13.532790 25691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:13.533139 25691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:13.534313 25691 hybrid_clock.cc:648] HybridClock initialized: now 1785977353534299 us; error 29 us; skew 500 ppm
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977353, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:13.536027 25691 init.cc:377] Logged in from keytab as oryx/127.15.147.188@KRBTEST.COM (short username oryx)
I20260806 00:49:13.536638 25691 webserver.cc:460] Webserver started at http://127.15.147.188:42785/ using document root <none> and password file <none>
I20260806 00:49:13.536897 25691 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:13.536967 25691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:13.538498 25691 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 00:49:13.539654 25960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:13.539922 25691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 00:49:13.540028 25691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/wal
uuid: "4358f4e37da5459594e616128989e93a"
format_stamp: "Formatted at 2026-08-06 00:49:13 on dist-test-slave-45dx"
server_key: "554e44c711aa5d6efecd0067a16e7d7c"
server_key_iv: "b1fa873028a053c067d1dc697abb1d37"
server_key_version: "encryptionkey@0"
I20260806 00:49:13.540143 25691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:13.549830 25691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:13.552470 25691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:13.558399 25691 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.188:34207
I20260806 00:49:13.558856 26031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.188:34207 every 8 connection(s)
I20260806 00:49:13.559211 25691 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/master-2/data/info.pb
I20260806 00:49:13.560927 26033 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:49:13.561833 26033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: Bootstrap starting.
I20260806 00:49:13.569489 25347 heartbeater.cc:503] Master 127.15.147.190:35655 was elected leader, sending a full tablet report...
I20260806 00:49:13.572036 26033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:49:13.572436 26033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: Bootstrap complete.
I20260806 00:49:13.572995 26033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: NON_VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: true } }
I20260806 00:49:13.573140 26033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4358f4e37da5459594e616128989e93a, State: Initialized, Role: LEARNER
I20260806 00:49:13.573275 26033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: NON_VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: true } }
I20260806 00:49:13.573662 26036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: NON_VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: true } } }
I20260806 00:49:13.573844 26033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:49:13.573865 26036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [sys.catalog]: This master's current role is: LEARNER
I20260806 00:49:13.577840 26050 catalog_manager.cc:1259] Loaded cluster ID: 6177584354ee43e8bdd6609d65673d28
I20260806 00:49:13.577970 26050 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: loading cluster ID for follower catalog manager: success
I20260806 00:49:13.579741 26050 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: acquiring CA information for follower catalog manager: success
I20260806 00:49:13.580989 26050 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 00:49:13.617751 25931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:13.618134 25931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:13.618198 25931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:13.624560 25931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:13.624650 25931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:13.624775 25931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:49:13.630781 25931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:36797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.129.keytab
--principal=oryx/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:35801
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=34231
--tserver_master_addrs=127.15.147.190:35655,127.15.147.189:33281,127.15.147.188:34207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:13.632017 25931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:13.633178 25931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:13.645311 26062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:13.645385 26058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:13.645582 26057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:13.645928 25931 server_base.cc:1034] running on GCE node
I20260806 00:49:13.646341 25931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:13.647051 25931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:13.648273 25931 hybrid_clock.cc:648] HybridClock initialized: now 1785977353648256 us; error 35 us; skew 500 ppm
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977353, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:13.651734 25931 init.cc:377] Logged in from keytab as oryx/127.15.147.129@KRBTEST.COM (short username oryx)
I20260806 00:49:13.652791 25931 webserver.cc:460] Webserver started at http://127.15.147.129:34231/ using document root <none> and password file <none>
I20260806 00:49:13.653434 25931 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:13.653537 25931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:13.658171 25931 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:49:13.660436 26072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:13.661789 25931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:49:13.661938 25931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad"
format_stamp: "Formatted at 2026-08-06 00:49:12 on dist-test-slave-45dx"
server_key: "0b8649932cd520a314cb97eed5371643"
server_key_iv: "895961716551032035356923ede9b67b"
server_key_version: "encryptionkey@0"
I20260806 00:49:13.662611 25931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:13.688308 25931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:13.691617 25931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:13.691838 25931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:13.692570 25931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:13.693507 25931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:13.693579 25931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:13.693653 25931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:13.693706 25931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:13.712994 25931 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:35801
I20260806 00:49:13.713162 26233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:35801 every 8 connection(s)
I20260806 00:49:13.714138 25931 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:49:13.716862 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 25931
I20260806 00:49:13.717090 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 25164
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977353, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.188@KRBTEST.COM
I20260806 00:49:13.733858 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:45675
--local_ip_for_outbound_sockets=127.15.147.130
--tserver_master_addrs=127.15.147.190:35655
--webserver_port=37489
--webserver_interface=127.15.147.130
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.130.keytab
--principal=oryx/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:36797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.147.190:35655,127.15.147.189:33281
--tserver_master_addrs=127.15.147.190:35655,127.15.147.189:33281,127.15.147.188:34207 with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977353, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.189@KRBTEST.COM
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977353, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM
I20260806 00:49:13.739894 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.716201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:47499 (local address 127.15.147.190:35655)
0806 00:49:13.716428 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.716435 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.723678 (+ 7243us) server_negotiation.cc:407] Connection header received
0806 00:49:13.723830 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.723835 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.723902 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.724014 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.726598 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.727457 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.729955 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.730289 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.730393 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.735317 (+ 4924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.735323 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.735329 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.735367 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.737504 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.738191 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.738195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.738197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.738291 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.738655 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.738661 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.738666 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.738950 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.739039 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.739497 (+ 458us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.739638 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 00:49:13.739935 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.718166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:51749 (local address 127.15.147.188:34207)
0806 00:49:13.724486 (+ 6320us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.724490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.724983 (+ 493us) server_negotiation.cc:407] Connection header received
0806 00:49:13.725037 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.725060 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.725130 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.725224 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.726598 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.727773 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.729921 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.730373 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.730530 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.734475 (+ 3945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.734478 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.734481 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.734510 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.736589 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.737324 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.737329 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.737332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.737433 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.737882 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.737885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.737915 (+ 30us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.738201 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.738352 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.739658 (+ 1306us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.739786 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6218,"thread_start_us":1808,"threads_started":1}
I20260806 00:49:13.740674 26242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.716734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.129:47499)
0806 00:49:13.723522 (+ 6788us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.723541 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.723710 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.724349 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.724358 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.725238 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.726401 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.726425 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.727752 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.727755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.729774 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.729786 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.730629 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.730636 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.730906 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.731853 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.731876 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.735144 (+ 3268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.737635 (+ 2491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.737638 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.737640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.738068 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.738401 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.738405 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.738407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.738541 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.739128 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.739134 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.739301 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.740148 (+ 847us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.740357 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6678,"mutex_wait_us":448,"spinlock_wait_cycles":19200,"thread_start_us":320,"threads_started":1}
I20260806 00:49:13.740738 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.716395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:42145 (local address 127.15.147.189:33281)
0806 00:49:13.716531 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.716535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.724796 (+ 8261us) server_negotiation.cc:407] Connection header received
0806 00:49:13.724859 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.724864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.724928 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.725011 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.726718 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.727622 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.730021 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.730471 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.730577 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.735252 (+ 4675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.735258 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.735262 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.735296 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.737521 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.738184 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.738188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.738190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.738308 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.738602 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.738605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.738607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.738811 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.738923 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.740452 (+ 1529us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.740570 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 00:49:13.740921 26243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.716721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:33281 (local address 127.15.147.129:42145)
0806 00:49:13.720437 (+ 3716us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.720513 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.722903 (+ 2390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.725235 (+ 2332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.725241 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.726194 (+ 953us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.726552 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.726565 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.727783 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.727787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.729863 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.729879 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.730822 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.730828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.731032 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.731953 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.731969 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.734731 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.737635 (+ 2904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.737638 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.737640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.738068 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.738410 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.738413 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.738414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.738501 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.738985 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.738987 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.739074 (+ 87us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.739162 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.739297 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3393,"mutex_wait_us":593,"spinlock_wait_cycles":4480,"thread_start_us":342,"threads_started":1}
I20260806 00:49:13.741271 26240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.718040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34207 (local address 127.15.147.129:51749)
0806 00:49:13.724692 (+ 6652us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.724706 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.724864 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.725400 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.725403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.725814 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.726401 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.726425 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.727967 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.727970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.729774 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.729786 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.730688 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.730691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.730906 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.732046 (+ 1140us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.732061 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.734287 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.736726 (+ 2439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.736734 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.736751 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.737149 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.737562 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.737564 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.737568 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.737775 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.738318 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.738326 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.738655 (+ 329us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.738879 (+ 224us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.739239 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6582,"mutex_wait_us":249,"spinlock_wait_cycles":2432,"thread_start_us":134,"threads_started":1}
I20260806 00:49:13.742024 26236 heartbeater.cc:347] Connected to a master server at 127.15.147.190:35655
I20260806 00:49:13.742055 26237 heartbeater.cc:347] Connected to a master server at 127.15.147.189:33281
I20260806 00:49:13.742255 26236 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:13.742254 26237 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:13.742409 26235 heartbeater.cc:347] Connected to a master server at 127.15.147.188:34207
I20260806 00:49:13.742475 26235 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:13.742934 26237 heartbeater.cc:511] Master 127.15.147.189:33281 requested a full tablet report, sending...
I20260806 00:49:13.742936 26235 heartbeater.cc:511] Master 127.15.147.188:34207 requested a full tablet report, sending...
I20260806 00:49:13.743001 26236 heartbeater.cc:511] Master 127.15.147.190:35655 requested a full tablet report, sending...
I20260806 00:49:13.744294 24871 ts_manager.cc:194] Re-registered known tserver with Master: e9c22b36dc1d43afbe4f6d3e2d2592ad (127.15.147.129:35801)
I20260806 00:49:13.744585 25725 ts_manager.cc:194] Registered new tserver with Master: e9c22b36dc1d43afbe4f6d3e2d2592ad (127.15.147.129:35801)
I20260806 00:49:13.745065 24871 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:47499
I20260806 00:49:13.745282 25983 ts_manager.cc:194] Registered new tserver with Master: e9c22b36dc1d43afbe4f6d3e2d2592ad (127.15.147.129:35801)
I20260806 00:49:13.806382 25532 heartbeater.cc:503] Master 127.15.147.190:35655 was elected leader, sending a full tablet report...
W20260806 00:49:13.858111 26246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:13.858408 26246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:13.858481 26246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:13.862422 26246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:13.862515 26246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:13.862613 26246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:49:13.867743 26246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:36797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.130.keytab
--principal=oryx/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:45675
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=37489
--tserver_master_addrs=127.15.147.190:35655,127.15.147.189:33281,127.15.147.188:34207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:13.868948 26246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:13.869985 26246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:13.880112 26252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:13.880160 26253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:13.880133 26255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:13.880287 26246 server_base.cc:1034] running on GCE node
I20260806 00:49:13.880962 26246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:13.881589 26246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:13.882783 26246 hybrid_clock.cc:648] HybridClock initialized: now 1785977353882765 us; error 33 us; skew 500 ppm
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977353, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:13.885573 26246 init.cc:377] Logged in from keytab as oryx/127.15.147.130@KRBTEST.COM (short username oryx)
I20260806 00:49:13.886339 26246 webserver.cc:460] Webserver started at http://127.15.147.130:37489/ using document root <none> and password file <none>
I20260806 00:49:13.886847 26246 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:13.886915 26246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:13.890597 26246 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:49:13.892650 26267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:13.893889 26246 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:49:13.893986 26246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c"
format_stamp: "Formatted at 2026-08-06 00:49:12 on dist-test-slave-45dx"
server_key: "4f646fe4ab25674d8f99a86eae585b70"
server_key_iv: "08dffab1eccfa20794dc7c73490f5294"
server_key_version: "encryptionkey@0"
I20260806 00:49:13.894507 26246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:13.915839 25666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.912176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34207 (local address 127.0.0.1:53912)
0806 00:49:13.912376 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.912394 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.912506 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.912773 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.912775 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.912788 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:13.912873 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.912877 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.913846 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.913849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.914570 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.914578 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.915501 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.915503 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.915574 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.915596 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.915646 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.915710 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 00:49:13.915894 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.912346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53912 (local address 127.15.147.188:34207)
0806 00:49:13.912488 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.912493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.912512 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:49:13.912591 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.912594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.912656 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.912751 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:13.912987 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.913734 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.914693 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.915382 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.915506 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.915658 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.915689 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.915765 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:49:13.916800 25996 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 00:49:13.917357 26036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13ed091816be4886ac2042f64901402c. Latest consensus state: current_term: 1 leader_uuid: "13ed091816be4886ac2042f64901402c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: NON_VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: true } } }
I20260806 00:49:13.917533 26036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [sys.catalog]: This master's current role is: LEARNER
I20260806 00:49:13.921065 26246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:13.924007 26246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:13.924166 26246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:13.924765 26246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:13.925619 26246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:13.925675 26246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:13.925715 26246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:13.925738 26246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:13.938086 26246 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:45675
I20260806 00:49:13.938104 26424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:45675 every 8 connection(s)
I20260806 00:49:13.939288 26246 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:49:13.943949 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 26246
I20260806 00:49:13.944145 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 25354
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977353, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.188@KRBTEST.COM
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977353, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM
Aug 06 00:49:13 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977353, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.189@KRBTEST.COM
I20260806 00:49:13.958966 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:36609
--local_ip_for_outbound_sockets=127.15.147.131
--tserver_master_addrs=127.15.147.190:35655
--webserver_port=45799
--webserver_interface=127.15.147.131
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.131.keytab
--principal=oryx/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:36797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.147.190:35655,127.15.147.189:33281
--tserver_master_addrs=127.15.147.190:35655,127.15.147.189:33281,127.15.147.188:34207 with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:49:13.959753 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.943507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:34655 (local address 127.15.147.188:34207)
0806 00:49:13.943619 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.943623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.943647 (+ 24us) server_negotiation.cc:407] Connection header received
0806 00:49:13.944371 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.944378 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.944446 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.944516 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.946585 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.947365 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.949437 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.949743 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.949827 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.954984 (+ 5157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.954991 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.954995 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.955028 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.957017 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.957682 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.957688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.957692 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.957803 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.958290 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.958296 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.958300 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.958542 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.958623 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.959335 (+ 712us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.959480 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 00:49:13.959759 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.943138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:35863 (local address 127.15.147.190:35655)
0806 00:49:13.943283 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.943290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.943309 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:49:13.944367 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.944370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.944423 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.944492 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.947037 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.947784 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.949544 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.949828 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.949887 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.954452 (+ 4565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.954458 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.954464 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.954499 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.957195 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.957880 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.957883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.957885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.957972 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.958305 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.958309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.958311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.958533 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.958708 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.959456 (+ 748us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.959569 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:49:13.960382 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.941519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:40323 (local address 127.15.147.189:33281)
0806 00:49:13.941678 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:13.941683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:13.943025 (+ 1342us) server_negotiation.cc:407] Connection header received
0806 00:49:13.944262 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:13.944266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:13.944363 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:13.944423 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:13.950442 (+ 6019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.951250 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.952988 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.953388 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.953530 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.956354 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:13.956360 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:13.956364 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:13.956398 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:13.958216 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.958980 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.958983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.958985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.959069 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:13.959380 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:13.959385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:13.959390 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:13.959635 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:13.959706 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:13.960113 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 00:49:13.960223 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 00:49:13.962005 26433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.943119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34207 (local address 127.15.147.130:34655)
0806 00:49:13.943534 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.943545 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.944150 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.944792 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.944801 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.946064 (+ 1263us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.946384 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.946407 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.947605 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.947608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.949314 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.949324 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.950034 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.950038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.950395 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.951242 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.951263 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.954799 (+ 3536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.957160 (+ 2361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.957168 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.957187 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.957548 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.957939 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.957945 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.957950 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.958175 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.958705 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.958710 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.959048 (+ 338us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.960292 (+ 1244us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.960721 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":163,"spinlock_wait_cycles":4864,"thread_start_us":170,"threads_started":1}
I20260806 00:49:13.962268 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.941993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:33281 (local address 127.15.147.130:40323)
0806 00:49:13.942931 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.942945 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.944018 (+ 1073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.944788 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.944797 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.945538 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.949267 (+ 3729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.949279 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.951760 (+ 2481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.951763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.952814 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.952826 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.953550 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.953555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.953670 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.954219 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.954234 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.956153 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.958469 (+ 2316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.958471 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.958473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.958879 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.959196 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.959199 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.959200 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.959276 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.959796 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.959801 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.959893 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.960582 (+ 689us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.960869 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"mutex_wait_us":200,"spinlock_wait_cycles":6528,"thread_start_us":254,"threads_started":1}
I20260806 00:49:13.961985 26430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:13.941993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.130:35863)
0806 00:49:13.942767 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:13.942825 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:13.944018 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:13.944788 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:13.944797 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:13.945648 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:13.946384 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.946407 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.947982 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.947985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.949442 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:13.949448 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:13.950034 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:13.950038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:13.951391 (+ 1353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:13.952077 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:13.952097 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:13.954261 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:13.957344 (+ 3083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.957351 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.957357 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.957756 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.958093 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:13.958097 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:13.958110 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:13.958193 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:13.958783 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:13.958786 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:13.959030 (+ 244us) client_negotiation.cc:769] Sending connection context
0806 00:49:13.960291 (+ 1261us) client_negotiation.cc:240] Negotiation successful
0806 00:49:13.960810 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":1290,"spinlock_wait_cycles":9600,"thread_start_us":683,"threads_started":1}
I20260806 00:49:13.963546 26427 heartbeater.cc:347] Connected to a master server at 127.15.147.190:35655
I20260806 00:49:13.963570 26425 heartbeater.cc:347] Connected to a master server at 127.15.147.188:34207
I20260806 00:49:13.963586 26428 heartbeater.cc:347] Connected to a master server at 127.15.147.189:33281
I20260806 00:49:13.963841 26425 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:13.963841 26427 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:13.963841 26428 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:13.964450 26425 heartbeater.cc:511] Master 127.15.147.188:34207 requested a full tablet report, sending...
I20260806 00:49:13.964457 26428 heartbeater.cc:511] Master 127.15.147.189:33281 requested a full tablet report, sending...
I20260806 00:49:13.964457 26427 heartbeater.cc:511] Master 127.15.147.190:35655 requested a full tablet report, sending...
I20260806 00:49:13.965569 24871 ts_manager.cc:194] Re-registered known tserver with Master: 308ca6c0a0b7403ba8c7187f1fd4f98c (127.15.147.130:45675)
I20260806 00:49:13.965664 25983 ts_manager.cc:194] Registered new tserver with Master: 308ca6c0a0b7403ba8c7187f1fd4f98c (127.15.147.130:45675)
I20260806 00:49:13.966166 25725 ts_manager.cc:194] Registered new tserver with Master: 308ca6c0a0b7403ba8c7187f1fd4f98c (127.15.147.130:45675)
I20260806 00:49:13.966267 24871 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:35863
W20260806 00:49:14.080663 26434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:14.080927 26434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:14.080968 26434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:14.085500 26434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:14.085561 26434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:14.085618 26434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:49:14.090708 26434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:36797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.131.keytab
--principal=oryx/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:36609
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=45799
--tserver_master_addrs=127.15.147.190:35655,127.15.147.189:33281,127.15.147.188:34207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:14.091718 26434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:14.092752 26434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:14.102813 26440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:14.102861 26439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:14.103147 26434 server_base.cc:1034] running on GCE node
W20260806 00:49:14.102823 26442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:14.103559 26434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:14.104149 26434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:14.105322 26434 hybrid_clock.cc:648] HybridClock initialized: now 1785977354105309 us; error 36 us; skew 500 ppm
Aug 06 00:49:14 dist-test-slave-45dx krb5kdc[24806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977354, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:14.108028 26434 init.cc:377] Logged in from keytab as oryx/127.15.147.131@KRBTEST.COM (short username oryx)
I20260806 00:49:14.108997 26434 webserver.cc:460] Webserver started at http://127.15.147.131:45799/ using document root <none> and password file <none>
I20260806 00:49:14.109560 26434 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:14.109666 26434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:14.113597 26434 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 00:49:14.115727 26450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:14.116935 26434 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:49:14.117061 26434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "c84469497d4f45d6860a4c4603d326d0"
format_stamp: "Formatted at 2026-08-06 00:49:12 on dist-test-slave-45dx"
server_key: "92236056a89fb422180efc6dc2c58e04"
server_key_iv: "73401d8350428bc3ee3e83f70a5cb56f"
server_key_version: "encryptionkey@0"
I20260806 00:49:14.117599 26434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:14.158905 26434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:14.162182 26434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:14.162393 26434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:14.163022 26434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:14.163887 26434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:14.163956 26434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:14.164026 26434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:14.164072 26434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:14.174422 26434 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:36609
I20260806 00:49:14.174460 26634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:36609 every 8 connection(s)
I20260806 00:49:14.175603 26434 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:49:14.179672 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 26434
Aug 06 00:49:14 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977354, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.188@KRBTEST.COM
Aug 06 00:49:14 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977354, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.189@KRBTEST.COM
Aug 06 00:49:14 dist-test-slave-45dx krb5kdc[24806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977354, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM
I20260806 00:49:14.196327 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.177560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:56247 (local address 127.15.147.188:34207)
0806 00:49:14.177700 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:14.177704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:14.178999 (+ 1295us) server_negotiation.cc:407] Connection header received
0806 00:49:14.180444 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:14.180448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:14.180515 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:14.180569 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:14.183819 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.184599 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.185965 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.186301 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.186378 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.191390 (+ 5012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:14.191395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:14.191398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:14.191426 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:14.193390 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.194133 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.194139 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.194141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.194236 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.194722 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.194726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.194728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.195009 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:14.195093 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:14.195943 (+ 850us) server_negotiation.cc:299] Negotiation successful
0806 00:49:14.196087 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:49:14.198433 26656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.182073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37110 (local address 127.15.147.189:33281)
0806 00:49:14.183650 (+ 1577us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:14.183655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:14.185054 (+ 1399us) server_negotiation.cc:407] Connection header received
0806 00:49:14.185095 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:14.185098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:14.185142 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:14.185220 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:14.189607 (+ 4387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.190430 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.191744 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.192102 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.192195 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.193927 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:14.193932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:14.193934 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:14.193962 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:14.195830 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.196722 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.196726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.196727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.196823 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.197147 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.197150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.197152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.197363 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:14.197511 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:14.198156 (+ 645us) server_negotiation.cc:299] Negotiation successful
0806 00:49:14.198276 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1488,"thread_start_us":73,"threads_started":1}
I20260806 00:49:14.198446 26653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.181872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33860 (local address 127.15.147.190:35655)
0806 00:49:14.184821 (+ 2949us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:14.184826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:14.184839 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:49:14.184892 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:14.184895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:14.184947 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:14.186727 (+ 1780us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:14.188038 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.188820 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.190068 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.190453 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.190551 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.193481 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:14.193487 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:14.193491 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:14.193519 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:14.195749 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.196409 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.196412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.196414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.196509 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.196867 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.196871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.196872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.197124 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:14.197249 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:14.198014 (+ 765us) server_negotiation.cc:299] Negotiation successful
0806 00:49:14.198145 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2877,"thread_start_us":107,"threads_started":1}
I20260806 00:49:14.199698 26641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.177945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34207 (local address 127.15.147.131:56247)
0806 00:49:14.178827 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:14.178868 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:14.180174 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:14.180894 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:14.180902 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:14.181598 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:14.183664 (+ 2066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:14.183672 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.184733 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.184736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:14.185823 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:14.185834 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.186599 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.186602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:14.187011 (+ 409us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.188230 (+ 1219us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:14.188244 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:14.191178 (+ 2934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:14.193547 (+ 2369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:14.193555 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:14.193578 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:14.193992 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:14.194360 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:14.194369 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:14.194371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:14.194592 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:14.195192 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:14.195203 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:14.195558 (+ 355us) client_negotiation.cc:769] Sending connection context
0806 00:49:14.196587 (+ 1029us) client_negotiation.cc:240] Negotiation successful
0806 00:49:14.197054 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"mutex_wait_us":334,"spinlock_wait_cycles":3840,"thread_start_us":537,"threads_started":1}
I20260806 00:49:14.199951 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.179418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:59693 (local address 127.15.147.190:35655)
0806 00:49:14.179551 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:14.179557 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:14.180078 (+ 521us) server_negotiation.cc:407] Connection header received
0806 00:49:14.180462 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:14.180465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:14.180515 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:14.180569 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:14.183055 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.183896 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.185967 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.186308 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.186388 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.193135 (+ 6747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:14.193138 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:14.193141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:14.193170 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:14.197771 (+ 4601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.198606 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.198609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.198611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.198741 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.199038 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.199043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.199044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.199267 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:14.199364 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:14.199695 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 00:49:14.199821 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:49:14.200481 26642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.179285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.131:59693)
0806 00:49:14.179968 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:14.179984 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:14.180174 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:14.180893 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:14.180902 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:14.182183 (+ 1281us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:14.182865 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:14.182889 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.184088 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.184091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:14.185832 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:14.185839 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.186599 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.186602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:14.187006 (+ 404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.188742 (+ 1736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:14.188756 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:14.192551 (+ 3795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:14.197902 (+ 5351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:14.197904 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:14.197906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:14.198148 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:14.198858 (+ 710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:14.198860 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:14.198861 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:14.198932 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:14.199405 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:14.199408 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:14.199514 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 00:49:14.200113 (+ 599us) client_negotiation.cc:240] Negotiation successful
0806 00:49:14.200263 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"mutex_wait_us":313,"spinlock_wait_cycles":896,"thread_start_us":417,"threads_started":1}
I20260806 00:49:14.201030 26643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.179285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:33281 (local address 127.15.147.131:41399)
0806 00:49:14.180209 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:14.180221 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:14.180296 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:14.181012 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:14.181015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:14.181611 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:14.182862 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:14.182890 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.184080 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.184083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:14.185805 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:14.185816 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.186627 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.186632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:14.187008 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.188023 (+ 1015us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:14.188044 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:14.192951 (+ 4907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:14.198264 (+ 5313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:14.198267 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:14.198268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:14.198491 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:14.199289 (+ 798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:14.199294 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:14.199299 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:14.199391 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:14.199846 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:14.199849 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:14.199951 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 00:49:14.200472 (+ 521us) client_negotiation.cc:240] Negotiation successful
0806 00:49:14.200644 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"mutex_wait_us":1185,"thread_start_us":90,"threads_started":1}
I20260806 00:49:14.201324 26635 heartbeater.cc:347] Connected to a master server at 127.15.147.188:34207
I20260806 00:49:14.201632 26635 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:14.201712 26636 heartbeater.cc:347] Connected to a master server at 127.15.147.190:35655
I20260806 00:49:14.201828 26636 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:14.200430 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.179504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:41399 (local address 127.15.147.189:33281)
0806 00:49:14.180638 (+ 1134us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:14.180644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:14.180661 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:49:14.180714 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:14.180721 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:14.180783 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:14.180872 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:14.183068 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.183896 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.185945 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.186330 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.186425 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.195724 (+ 9299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:14.195728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:14.195730 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:14.195757 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:14.197533 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.199077 (+ 1544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.199083 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.199087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.199175 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.199492 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.199495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.199497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.199709 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:14.199788 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:14.200162 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 00:49:14.200263 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1030}
I20260806 00:49:14.202298 26635 heartbeater.cc:511] Master 127.15.147.188:34207 requested a full tablet report, sending...
I20260806 00:49:14.202301 26636 heartbeater.cc:511] Master 127.15.147.190:35655 requested a full tablet report, sending...
I20260806 00:49:14.202443 26639 heartbeater.cc:347] Connected to a master server at 127.15.147.189:33281
I20260806 00:49:14.202505 26639 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:14.202636 26639 heartbeater.cc:511] Master 127.15.147.189:33281 requested a full tablet report, sending...
I20260806 00:49:14.203450 24870 ts_manager.cc:194] Re-registered known tserver with Master: c84469497d4f45d6860a4c4603d326d0 (127.15.147.131:36609)
I20260806 00:49:14.204209 24870 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:59693
I20260806 00:49:14.204715 25725 ts_manager.cc:194] Registered new tserver with Master: c84469497d4f45d6860a4c4603d326d0 (127.15.147.131:36609)
I20260806 00:49:14.205322 25983 ts_manager.cc:194] Registered new tserver with Master: c84469497d4f45d6860a4c4603d326d0 (127.15.147.131:36609)
I20260806 00:49:14.202615 24871 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:49:14.206975 24871 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:49:14.207043 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.182476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53926 (local address 127.15.147.188:34207)
0806 00:49:14.183953 (+ 1477us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:14.183959 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:14.184666 (+ 707us) server_negotiation.cc:407] Connection header received
0806 00:49:14.185475 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:14.185478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:14.185535 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:14.185614 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:14.189568 (+ 3954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.190331 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.191561 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.191906 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.192001 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.203061 (+ 11060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:14.203065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:14.203067 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:14.203095 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:14.204842 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.205386 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.205389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.205391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.205482 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.205787 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.205790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.205792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.206007 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:14.206128 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:14.206734 (+ 606us) server_negotiation.cc:299] Negotiation successful
0806 00:49:14.206870 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1380,"thread_start_us":74,"threads_started":1}
I20260806 00:49:14.209672 25934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13ed091816be4886ac2042f64901402c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: NON_VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: true } } }
I20260806 00:49:14.209822 25934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:14.220960 25934 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c: attempting to promote NON_VOTER 4358f4e37da5459594e616128989e93a to VOTER
I20260806 00:49:14.221302 25934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false } }
I20260806 00:49:14.222900 25736 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 1 FOLLOWER]: Refusing update from remote peer 13ed091816be4886ac2042f64901402c: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260806 00:49:14.223486 25994 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [term 1 LEARNER]: Refusing update from remote peer 13ed091816be4886ac2042f64901402c: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260806 00:49:14.224136 26659 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260806 00:49:14.224798 26659 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 00:49:14.227625 26673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.219587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:36609 (local address 127.0.0.1:54550)
0806 00:49:14.220870 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:14.220891 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:14.221045 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:14.222922 (+ 1877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:14.222925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:14.222942 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:14.223034 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:14.223039 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.224455 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.224458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:14.225761 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:14.225773 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.227178 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.227184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:14.227277 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.227307 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 00:49:14.227394 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 00:49:14.227483 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1221,"mutex_wait_us":52,"thread_start_us":89,"threads_started":1}
I20260806 00:49:14.227938 26661 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 4358f4e37da5459594e616128989e93a (127.15.147.188) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false } } }
I20260806 00:49:14.228116 26672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.219129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:45675 (local address 127.0.0.1:49268)
0806 00:49:14.220042 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:14.220058 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:14.220181 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:14.221699 (+ 1518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:14.221702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:14.221722 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:14.222285 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:14.222294 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.224127 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.224131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:14.225771 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:14.225777 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.227531 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.227534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:14.227617 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.227645 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 00:49:14.227912 (+ 267us) client_negotiation.cc:240] Negotiation successful
0806 00:49:14.227989 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"mutex_wait_us":64,"thread_start_us":200,"threads_started":1}
I20260806 00:49:14.228180 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.220453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54550 (local address 127.15.147.131:36609)
0806 00:49:14.222137 (+ 1684us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:14.222196 (+ 59us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:14.222221 (+ 25us) server_negotiation.cc:407] Connection header received
0806 00:49:14.222295 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:14.222300 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:14.222513 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:14.222753 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:14.223179 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.224299 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.225908 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.226840 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.226964 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.227108 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:14.227899 (+ 791us) server_negotiation.cc:299] Negotiation successful
0806 00:49:14.228005 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1578,"thread_start_us":142,"threads_started":1}
I20260806 00:49:14.228788 26674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.220157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49268 (local address 127.15.147.130:45675)
0806 00:49:14.220755 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:14.220806 (+ 51us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:14.220831 (+ 25us) server_negotiation.cc:407] Connection header received
0806 00:49:14.220916 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:14.220944 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:14.221313 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:14.221729 (+ 416us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:14.222424 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.224002 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.225916 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.226835 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.226964 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.228485 (+ 1521us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:14.228563 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 00:49:14.228656 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":210,"threads_started":1}
I20260806 00:49:14.229460 26661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "13ed091816be4886ac2042f64901402c" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false } } }
I20260806 00:49:14.229578 26661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:14.229713 26661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13ed091816be4886ac2042f64901402c" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false } } }
I20260806 00:49:14.229805 26661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:14.229955 26661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13ed091816be4886ac2042f64901402c" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false } } }
I20260806 00:49:14.230050 26661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:14.230520 25996 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.8->[1.9-1.9] Dedup: 1.9->[]
I20260806 00:49:14.230571 25736 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 4358f4e37da5459594e616128989e93a (127.15.147.188) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false } } }
I20260806 00:49:14.231662 26529 tablet_service.cc:1467] Processing CreateTablet for tablet d226bddb102c4a9a929992daf0ad6f95 (DEFAULT_TABLE table=test-table [id=027111e312454abeb47a979252bfee7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:14.232883 26529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d226bddb102c4a9a929992daf0ad6f95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:14.232631 25994 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 4358f4e37da5459594e616128989e93a (127.15.147.188) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false } } }
I20260806 00:49:14.234143 25666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.218523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:35801 (local address 127.0.0.1:41014)
0806 00:49:14.218664 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:14.218677 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:14.218742 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:14.219616 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:14.219619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:14.219638 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:14.219741 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:14.219748 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.231757 (+ 12009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.231761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:14.232710 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:14.232719 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.233794 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.233797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:14.233876 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.233897 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 00:49:14.233959 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 00:49:14.234031 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 00:49:14.234230 26671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.218705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41014 (local address 127.15.147.129:35801)
0806 00:49:14.219125 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:14.219143 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:14.219165 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:49:14.219253 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:14.219258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:14.219457 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:14.219905 (+ 448us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:14.220051 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.231617 (+ 11566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.232992 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.233669 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.233807 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.233952 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:14.234001 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 00:49:14.234086 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":205,"threads_started":1}
I20260806 00:49:14.234797 26036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "13ed091816be4886ac2042f64901402c" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false } } }
I20260806 00:49:14.234917 26036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:49:14.235333 26677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "13ed091816be4886ac2042f64901402c" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false } } }
I20260806 00:49:14.235492 26677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:49:14.237308 26138 tablet_service.cc:1467] Processing CreateTablet for tablet d226bddb102c4a9a929992daf0ad6f95 (DEFAULT_TABLE table=test-table [id=027111e312454abeb47a979252bfee7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:14.238354 26138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d226bddb102c4a9a929992daf0ad6f95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:14.238682 26333 tablet_service.cc:1467] Processing CreateTablet for tablet d226bddb102c4a9a929992daf0ad6f95 (DEFAULT_TABLE table=test-table [id=027111e312454abeb47a979252bfee7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:14.239823 26333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d226bddb102c4a9a929992daf0ad6f95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:14.244590 26679 tablet_bootstrap.cc:492] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0: Bootstrap starting.
I20260806 00:49:14.246439 26680 tablet_bootstrap.cc:492] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad: Bootstrap starting.
I20260806 00:49:14.247602 26679 tablet_bootstrap.cc:654] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:14.248837 26680 tablet_bootstrap.cc:654] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:14.248919 26679 log.cc:824] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:14.249967 26680 log.cc:824] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:14.251157 26679 tablet_bootstrap.cc:492] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0: No bootstrap required, opened a new log
I20260806 00:49:14.251359 26679 ts_tablet_manager.cc:1397] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 00:49:14.251899 26680 tablet_bootstrap.cc:492] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad: No bootstrap required, opened a new log
I20260806 00:49:14.252092 26680 ts_tablet_manager.cc:1397] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:49:14.249951 26682 tablet_bootstrap.cc:492] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c: Bootstrap starting.
I20260806 00:49:14.255513 26682 tablet_bootstrap.cc:654] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:14.256543 26680 raft_consensus.cc:348] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } } peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } }
I20260806 00:49:14.256809 26680 raft_consensus.cc:374] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:14.256888 26680 raft_consensus.cc:729] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9c22b36dc1d43afbe4f6d3e2d2592ad, State: Initialized, Role: FOLLOWER
I20260806 00:49:14.257092 26682 log.cc:824] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:14.257390 26680 consensus_queue.cc:260] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } } peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } }
I20260806 00:49:14.258531 26680 ts_tablet_manager.cc:1428] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260806 00:49:14.258653 26679 raft_consensus.cc:348] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } } peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } }
I20260806 00:49:14.258977 26679 raft_consensus.cc:374] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:14.259030 26679 raft_consensus.cc:729] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84469497d4f45d6860a4c4603d326d0, State: Initialized, Role: FOLLOWER
I20260806 00:49:14.259431 26236 heartbeater.cc:503] Master 127.15.147.190:35655 was elected leader, sending a full tablet report...
I20260806 00:49:14.259663 26682 tablet_bootstrap.cc:492] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c: No bootstrap required, opened a new log
I20260806 00:49:14.259856 26682 ts_tablet_manager.cc:1397] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260806 00:49:14.261982 26679 consensus_queue.cc:260] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } } peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } }
I20260806 00:49:14.262771 26636 heartbeater.cc:503] Master 127.15.147.190:35655 was elected leader, sending a full tablet report...
I20260806 00:49:14.265180 26679 ts_tablet_manager.cc:1428] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0: Time spent starting tablet: real 0.014s user 0.010s sys 0.000s
I20260806 00:49:14.267621 26682 raft_consensus.cc:348] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } } peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } }
I20260806 00:49:14.267892 26682 raft_consensus.cc:374] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:14.267949 26682 raft_consensus.cc:729] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 308ca6c0a0b7403ba8c7187f1fd4f98c, State: Initialized, Role: FOLLOWER
I20260806 00:49:14.268405 26682 consensus_queue.cc:260] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } } peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } }
I20260806 00:49:14.269482 26682 ts_tablet_manager.cc:1428] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260806 00:49:14.269979 26427 heartbeater.cc:503] Master 127.15.147.190:35655 was elected leader, sending a full tablet report...
W20260806 00:49:14.426928 26640 tablet.cc:2369] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:49:14.441823 26429 tablet.cc:2369] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:49:14.456344 26687 raft_consensus.cc:482] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:49:14.456636 26687 raft_consensus.cc:504] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } } peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } }
I20260806 00:49:14.457932 26687 leader_election.cc:288] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9c22b36dc1d43afbe4f6d3e2d2592ad (127.15.147.129:35801), c84469497d4f45d6860a4c4603d326d0 (127.15.147.131:36609)
I20260806 00:49:14.462206 26430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.458159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:35801 (local address 127.15.147.130:42341)
0806 00:49:14.458356 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:14.458373 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:14.458476 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:14.458785 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:14.458788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:14.458807 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:14.459009 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:14.459017 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.459994 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.460000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:14.460951 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:14.460961 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.461708 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.461712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:14.461845 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.461905 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 00:49:14.461981 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 00:49:14.462064 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":20}
I20260806 00:49:14.462270 26671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.458299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:42341 (local address 127.15.147.129:35801)
0806 00:49:14.458436 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:14.458440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:14.458459 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:49:14.458609 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:14.458614 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:14.458680 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:14.458755 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:14.459154 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.459876 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.461078 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.461590 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.461685 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.461728 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:14.462011 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 00:49:14.462114 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:49:14.462512 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.458291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:36609 (local address 127.15.147.130:54995)
0806 00:49:14.458462 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:14.458477 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:14.458595 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:14.458905 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:14.458908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:14.458925 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:14.459157 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:14.459166 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.460078 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.460080 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:14.461224 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:14.461235 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.462006 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.462010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:14.462189 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.462213 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 00:49:14.462295 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 00:49:14.462370 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":54}
I20260806 00:49:14.462875 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.458344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:54995 (local address 127.15.147.131:36609)
0806 00:49:14.458502 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:14.458509 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:14.458528 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:49:14.458711 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:14.458714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:14.458782 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:14.458861 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:14.459292 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.459917 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.461359 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.461855 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.461993 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.462043 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:14.462694 (+ 651us) server_negotiation.cc:299] Negotiation successful
0806 00:49:14.462778 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 00:49:14.462920 26169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d226bddb102c4a9a929992daf0ad6f95" candidate_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" is_pre_election: true
I20260806 00:49:14.463294 26169 raft_consensus.cc:2454] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 308ca6c0a0b7403ba8c7187f1fd4f98c in term 0.
I20260806 00:49:14.463579 26564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d226bddb102c4a9a929992daf0ad6f95" candidate_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84469497d4f45d6860a4c4603d326d0" is_pre_election: true
I20260806 00:49:14.463860 26270 leader_election.cc:302] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 308ca6c0a0b7403ba8c7187f1fd4f98c, e9c22b36dc1d43afbe4f6d3e2d2592ad; no voters:
I20260806 00:49:14.463899 26564 raft_consensus.cc:2454] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 308ca6c0a0b7403ba8c7187f1fd4f98c in term 0.
I20260806 00:49:14.464190 26687 raft_consensus.cc:2781] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:49:14.464325 26687 raft_consensus.cc:482] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:49:14.464391 26687 raft_consensus.cc:3037] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:14.465603 26687 raft_consensus.cc:504] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } } peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } }
I20260806 00:49:14.465981 26687 leader_election.cc:288] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [CANDIDATE]: Term 1 election: Requested vote from peers e9c22b36dc1d43afbe4f6d3e2d2592ad (127.15.147.129:35801), c84469497d4f45d6860a4c4603d326d0 (127.15.147.131:36609)
I20260806 00:49:14.466455 26564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d226bddb102c4a9a929992daf0ad6f95" candidate_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84469497d4f45d6860a4c4603d326d0"
I20260806 00:49:14.466516 26169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d226bddb102c4a9a929992daf0ad6f95" candidate_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad"
I20260806 00:49:14.466585 26564 raft_consensus.cc:3037] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:14.466624 26169 raft_consensus.cc:3037] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 0 FOLLOWER]: Advancing to term 1
W20260806 00:49:14.467025 26238 tablet.cc:2369] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:49:14.468510 26169 raft_consensus.cc:2454] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 308ca6c0a0b7403ba8c7187f1fd4f98c in term 1.
I20260806 00:49:14.468510 26564 raft_consensus.cc:2454] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 308ca6c0a0b7403ba8c7187f1fd4f98c in term 1.
I20260806 00:49:14.468815 26270 leader_election.cc:302] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 308ca6c0a0b7403ba8c7187f1fd4f98c, e9c22b36dc1d43afbe4f6d3e2d2592ad; no voters:
I20260806 00:49:14.469085 26687 raft_consensus.cc:2781] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:14.469296 26687 raft_consensus.cc:686] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 1 LEADER]: Becoming Leader. State: Replica: 308ca6c0a0b7403ba8c7187f1fd4f98c, State: Running, Role: LEADER
I20260806 00:49:14.469645 26687 consensus_queue.cc:237] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } } peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } }
I20260806 00:49:14.473171 24871 catalog_manager.cc:5495] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c reported cstate change: term changed from 0 to 1, leader changed from <none> to 308ca6c0a0b7403ba8c7187f1fd4f98c (127.15.147.130). New cstate: current_term: 1 leader_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } health_report { overall_health: UNKNOWN } } }
I20260806 00:49:14.518271 26674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.514315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49272 (local address 127.15.147.130:45675)
0806 00:49:14.514464 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:14.514468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:14.514483 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:49:14.514655 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:14.514658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:14.514716 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:14.514825 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:49:14.515195 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.515850 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.516767 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.516989 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.517092 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.517477 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:49:14.517873 (+ 396us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:49:14.518008 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:14.518068 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 00:49:14.518127 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:49:14.524650 26564 raft_consensus.cc:1261] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0 [term 1 FOLLOWER]: Refusing update from remote peer 308ca6c0a0b7403ba8c7187f1fd4f98c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:14.524655 26169 raft_consensus.cc:1261] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 1 FOLLOWER]: Refusing update from remote peer 308ca6c0a0b7403ba8c7187f1fd4f98c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:14.525430 26687 consensus_queue.cc:1035] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:14.525667 26689 consensus_queue.cc:1035] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:14.533360 26693 mvcc.cc:205] Tried to move back new op lower bound from 7315363244125184000 to 7315363243910561792. Current Snapshot: MvccSnapshot[applied={T|T < 7315363244125184000}]
I20260806 00:49:14.535868 26694 mvcc.cc:205] Tried to move back new op lower bound from 7315363244125184000 to 7315363243910561792. Current Snapshot: MvccSnapshot[applied={T|T < 7315363244125184000}]
I20260806 00:49:14.538307 26696 mvcc.cc:205] Tried to move back new op lower bound from 7315363244125184000 to 7315363243910561792. Current Snapshot: MvccSnapshot[applied={T|T < 7315363244125184000}]
I20260806 00:49:14.545778 24870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:49:14.545957 24870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:49:14.550335 24870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 13ed091816be4886ac2042f64901402c: Sending DeleteTablet for 3 replicas of tablet d226bddb102c4a9a929992daf0ad6f95
I20260806 00:49:14.551222 26138 tablet_service.cc:1514] Processing DeleteTablet for tablet d226bddb102c4a9a929992daf0ad6f95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:14 UTC) from {username='oryx', principal='oryx/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:41014
I20260806 00:49:14.551311 26333 tablet_service.cc:1514] Processing DeleteTablet for tablet d226bddb102c4a9a929992daf0ad6f95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:14 UTC) from {username='oryx', principal='oryx/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:49268
I20260806 00:49:14.551844 26529 tablet_service.cc:1514] Processing DeleteTablet for tablet d226bddb102c4a9a929992daf0ad6f95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:14 UTC) from {username='oryx', principal='oryx/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:54550
I20260806 00:49:14.551977 26706 tablet_replica.cc:330] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad: stopping tablet replica
I20260806 00:49:14.552258 26706 raft_consensus.cc:2229] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:14.552410 26707 tablet_replica.cc:330] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0: stopping tablet replica
I20260806 00:49:14.552611 26706 raft_consensus.cc:2258] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:14.552770 26707 raft_consensus.cc:2229] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:14.553910 26707 raft_consensus.cc:2258] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:14.554087 26706 ts_tablet_manager.cc:1905] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:14.554669 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 24825
I20260806 00:49:14.562970 26707 ts_tablet_manager.cc:1905] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:14.564709 26706 ts_tablet_manager.cc:1918] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:49:14.564826 26706 log.cc:1196] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-0/wal/wals/d226bddb102c4a9a929992daf0ad6f95
I20260806 00:49:14.565196 26706 ts_tablet_manager.cc:1939] T d226bddb102c4a9a929992daf0ad6f95 P e9c22b36dc1d43afbe4f6d3e2d2592ad: Deleting consensus metadata
I20260806 00:49:14.565827 26707 ts_tablet_manager.cc:1918] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:49:14.565953 26707 log.cc:1196] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/d226bddb102c4a9a929992daf0ad6f95
I20260806 00:49:14.566262 26707 ts_tablet_manager.cc:1939] T d226bddb102c4a9a929992daf0ad6f95 P c84469497d4f45d6860a4c4603d326d0: Deleting consensus metadata
W20260806 00:49:14.570936 26454 connection.cc:535] server connection from 127.0.0.1:54550 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 00:49:14.571928 26076 connection.cc:535] server connection from 127.0.0.1:41014 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 00:49:14.580655 26656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.553377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37114 (local address 127.15.147.189:33281)
0806 00:49:14.553490 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:14.553496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:14.553510 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:49:14.553548 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:14.553551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:14.553598 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:14.553679 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:14.567887 (+ 14208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.568909 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.570148 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.570584 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.570720 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.573140 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:14.573143 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:14.573146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:14.573173 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:14.574920 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.575471 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.575477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.575482 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.575583 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.575946 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.575950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.575952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.576263 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:14.576457 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:14.580388 (+ 3931us) server_negotiation.cc:299] Negotiation successful
0806 00:49:14.580503 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 00:49:14.582252 26705 tablet_replica.cc:330] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c: stopping tablet replica
I20260806 00:49:14.582706 26705 raft_consensus.cc:2229] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:49:14.583187 26705 raft_consensus.cc:2258] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:14.583978 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.573661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37124 (local address 127.15.147.189:33281)
0806 00:49:14.573784 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:14.573791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:14.574206 (+ 415us) server_negotiation.cc:407] Connection header received
0806 00:49:14.574356 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:14.574360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:14.574404 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:14.574448 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:14.575498 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.576446 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.577701 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.578039 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.578099 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.579757 (+ 1658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:14.579760 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:14.579763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:14.579787 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:14.581768 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.582604 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.582607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.582609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.582691 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.583003 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.583006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.583007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.583225 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:14.583330 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:14.583722 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 00:49:14.583829 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:49:14.585865 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.574340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53946 (local address 127.15.147.188:34207)
0806 00:49:14.574462 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:14.574466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:14.574965 (+ 499us) server_negotiation.cc:407] Connection header received
0806 00:49:14.575153 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:14.575156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:14.575205 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:14.575263 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:14.577243 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.578014 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.579241 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.579567 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.579640 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.582004 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:14.582007 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:14.582010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:14.582039 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:14.583726 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.584217 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.584220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.584222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.584322 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.584783 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.584787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.584790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.585040 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:14.585154 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:14.585563 (+ 409us) server_negotiation.cc:299] Negotiation successful
0806 00:49:14.585708 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:49:14.587713 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:14.553445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53934 (local address 127.15.147.188:34207)
0806 00:49:14.553553 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:14.553558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:14.577306 (+ 23748us) server_negotiation.cc:407] Connection header received
0806 00:49:14.577339 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:14.577341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:14.577392 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:14.577453 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:14.578234 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.579221 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.580287 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:14.580710 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:14.580830 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:14.583733 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:14.583736 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:14.583738 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:14.583763 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:14.585885 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.586569 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.586572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.586574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.586666 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:14.586966 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:14.586969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:14.586971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:14.587121 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:14.587247 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:14.587458 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 00:49:14.587570 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 00:49:14.589622 26705 ts_tablet_manager.cc:1905] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:14.592459 26705 ts_tablet_manager.cc:1918] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:49:14.592566 26705 log.cc:1196] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/d226bddb102c4a9a929992daf0ad6f95
I20260806 00:49:14.592895 26705 ts_tablet_manager.cc:1939] T d226bddb102c4a9a929992daf0ad6f95 P 308ca6c0a0b7403ba8c7187f1fd4f98c: Deleting consensus metadata
W20260806 00:49:14.593941 26272 connection.cc:441] server connection from 127.0.0.1:49268 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49268 (request call id 1) could send its response
I20260806 00:49:15.538539 26723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:15.537901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.131:55293)
0806 00:49:15.538306 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:15.538417 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":121,"threads_started":1}
W20260806 00:49:15.539191 26636 heartbeater.cc:644] Failed to heartbeat to 127.15.147.190:35655 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
I20260806 00:49:15.539811 26725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:15.539029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.129:37625)
0806 00:49:15.539558 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:15.539692 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":83,"threads_started":1}
W20260806 00:49:15.540097 26236 heartbeater.cc:644] Failed to heartbeat to 127.15.147.190:35655 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
I20260806 00:49:15.540220 26727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:15.539571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.130:35057)
0806 00:49:15.540014 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:15.540115 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":65,"threads_started":1}
W20260806 00:49:15.540503 26427 heartbeater.cc:644] Failed to heartbeat to 127.15.147.190:35655 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
I20260806 00:49:16.050966 26730 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 13ed091816be4886ac2042f64901402c)
I20260806 00:49:16.051215 26733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 13ed091816be4886ac2042f64901402c)
I20260806 00:49:16.051250 26730 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false } }
I20260806 00:49:16.051349 26733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false } }
I20260806 00:49:16.051919 26733 leader_election.cc:288] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13ed091816be4886ac2042f64901402c (127.15.147.190:35655), 4358f4e37da5459594e616128989e93a (127.15.147.188:34207)
I20260806 00:49:16.052564 26734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.052049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.0.0.1:34348)
0806 00:49:16.052361 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.052457 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":69,"threads_started":1}
W20260806 00:49:16.052786 25706 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
I20260806 00:49:16.053124 26739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.052498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.0.0.1:34364)
0806 00:49:16.052914 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.053028 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":78,"threads_started":1}
I20260806 00:49:16.053464 26730 leader_election.cc:288] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13ed091816be4886ac2042f64901402c (127.15.147.190:35655), 2a526db9ea2b4fdcad2f43cd537a87cd (127.15.147.189:33281)
W20260806 00:49:16.053561 25961 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
I20260806 00:49:16.053711 26734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.053432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.0.0.1:34380)
0806 00:49:16.053584 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.053633 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260806 00:49:16.053853 25706 leader_election.cc:334] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 13ed091816be4886ac2042f64901402c (127.15.147.190:35655): Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
I20260806 00:49:16.054636 26739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.054416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.0.0.1:34394)
0806 00:49:16.054513 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.054566 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 00:49:16.054845 25961 leader_election.cc:334] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 13ed091816be4886ac2042f64901402c (127.15.147.190:35655): Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
I20260806 00:49:16.056897 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.052148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34207 (local address 127.0.0.1:53960)
0806 00:49:16.052451 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.052466 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:16.052568 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:16.053064 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:16.053068 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:16.053090 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:16.053219 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:16.053227 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.054416 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.054422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:16.055584 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:16.055596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.056422 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.056427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:16.056514 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:16.056602 (+ 88us) client_negotiation.cc:769] Sending connection context
0806 00:49:16.056666 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 00:49:16.056767 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":56,"thread_start_us":58,"threads_started":1}
I20260806 00:49:16.056929 26738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.052359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53960 (local address 127.15.147.188:34207)
0806 00:49:16.052790 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:16.052795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:16.052809 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:49:16.052858 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:16.052864 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:16.052923 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:16.053337 (+ 414us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:16.053455 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.054301 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.055737 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.056253 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.056420 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:16.056474 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:16.056701 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 00:49:16.056785 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":351,"thread_start_us":59,"threads_started":1}
I20260806 00:49:16.057438 25994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4358f4e37da5459594e616128989e93a" is_pre_election: true
I20260806 00:49:16.057664 26741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.052990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:33281 (local address 127.0.0.1:37138)
0806 00:49:16.053654 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.053667 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:16.053772 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:16.054164 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:16.054167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:16.054187 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:16.054296 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:16.054309 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.055275 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.055278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:16.056251 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:16.056262 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.057217 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.057222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:16.057339 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:16.057363 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 00:49:16.057443 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 00:49:16.057526 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":1,"thread_start_us":54,"threads_started":1}
I20260806 00:49:16.057690 26742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.053241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37138 (local address 127.15.147.189:33281)
0806 00:49:16.053966 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:16.053974 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:16.053986 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:49:16.054021 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:16.054024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:16.054060 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:16.054139 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:16.054453 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.055165 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.056436 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.057016 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.057157 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:16.057203 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:16.057473 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 00:49:16.057537 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":661,"thread_start_us":115,"threads_started":1}
I20260806 00:49:16.057679 25994 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a526db9ea2b4fdcad2f43cd537a87cd in term 1.
I20260806 00:49:16.058061 25710 leader_election.cc:302] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a526db9ea2b4fdcad2f43cd537a87cd, 4358f4e37da5459594e616128989e93a; no voters: 13ed091816be4886ac2042f64901402c
I20260806 00:49:16.058248 26733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 00:49:16.058310 26733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 13ed091816be4886ac2042f64901402c)
I20260806 00:49:16.058338 26733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 1 FOLLOWER]: Advancing to term 2
I20260806 00:49:16.058552 25736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4358f4e37da5459594e616128989e93a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" is_pre_election: true
I20260806 00:49:16.059568 26733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false } }
I20260806 00:49:16.059815 25736 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4358f4e37da5459594e616128989e93a in current term 2: Already voted for candidate 2a526db9ea2b4fdcad2f43cd537a87cd in this term.
I20260806 00:49:16.059985 26733 leader_election.cc:288] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [CANDIDATE]: Term 2 election: Requested vote from peers 13ed091816be4886ac2042f64901402c (127.15.147.190:35655), 4358f4e37da5459594e616128989e93a (127.15.147.188:34207)
I20260806 00:49:16.060379 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.060104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.0.0.1:34408)
0806 00:49:16.060230 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.060286 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 00:49:16.060334 25994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4358f4e37da5459594e616128989e93a"
I20260806 00:49:16.060487 25994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [term 1 FOLLOWER]: Advancing to term 2
I20260806 00:49:16.060531 25964 leader_election.cc:302] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4358f4e37da5459594e616128989e93a; no voters: 13ed091816be4886ac2042f64901402c, 2a526db9ea2b4fdcad2f43cd537a87cd
I20260806 00:49:16.061111 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.060896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.0.0.1:34412)
0806 00:49:16.061014 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.061062 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 00:49:16.061239 25706 leader_election.cc:334] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 13ed091816be4886ac2042f64901402c (127.15.147.190:35655): Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
I20260806 00:49:16.061719 25994 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a526db9ea2b4fdcad2f43cd537a87cd in term 2.
I20260806 00:49:16.061831 26730 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:49:16.062105 25710 leader_election.cc:302] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a526db9ea2b4fdcad2f43cd537a87cd, 4358f4e37da5459594e616128989e93a; no voters: 13ed091816be4886ac2042f64901402c
I20260806 00:49:16.062328 26733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 2 FOLLOWER]: Leader election won for term 2
I20260806 00:49:16.062590 26733 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [term 2 LEADER]: Becoming Leader. State: Replica: 2a526db9ea2b4fdcad2f43cd537a87cd, State: Running, Role: LEADER
I20260806 00:49:16.062866 26733 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false } }
I20260806 00:49:16.063988 26747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a526db9ea2b4fdcad2f43cd537a87cd. Latest consensus state: current_term: 2 leader_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false } } }
I20260806 00:49:16.064095 26747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:16.064438 26750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:49:16.067346 26750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:49:16.067816 26750 catalog_manager.cc:1259] Loaded cluster ID: 6177584354ee43e8bdd6609d65673d28
I20260806 00:49:16.067886 26750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:49:16.068535 26750 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:49:16.068926 26750 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: Loaded TSK: 0
I20260806 00:49:16.069585 26750 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:49:16.108120 25724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:49:16.111248 25724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:49:16.114459 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.114192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.0.0.1:34424)
0806 00:49:16.114308 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.114390 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 00:49:16.114567 25994 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [term 2 FOLLOWER]: Refusing update from remote peer 2a526db9ea2b4fdcad2f43cd537a87cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 00:49:16.115154 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.114899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.0.0.1:34434)
0806 00:49:16.115002 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.115051 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 00:49:16.115067 26733 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 00:49:16.115299 25706 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd -> Peer 13ed091816be4886ac2042f64901402c (127.15.147.190:35655): Couldn't send request to peer 13ed091816be4886ac2042f64901402c. Status: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 00:49:16.116703 26730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a526db9ea2b4fdcad2f43cd537a87cd. Latest consensus state: current_term: 2 leader_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false } } }
I20260806 00:49:16.116815 26730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:49:16.117899 26733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false } } }
I20260806 00:49:16.118026 26733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:16.118441 26730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "13ed091816be4886ac2042f64901402c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35655 } } peers { permanent_uuid: "2a526db9ea2b4fdcad2f43cd537a87cd" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 33281 } attrs { promote: false } } peers { permanent_uuid: "4358f4e37da5459594e616128989e93a" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34207 } attrs { promote: false } } }
I20260806 00:49:16.118588 26730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4358f4e37da5459594e616128989e93a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:49:16.128986 26759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.125169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54558 (local address 127.15.147.131:36609)
0806 00:49:16.125416 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:16.125419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:16.125432 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:49:16.125507 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:16.125510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:16.125564 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:16.125667 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:16.125912 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.126571 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.127948 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.128434 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.128574 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:16.128629 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:16.128793 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 00:49:16.128873 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":98,"threads_started":1}
I20260806 00:49:16.128985 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.124649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:36609 (local address 127.0.0.1:54558)
0806 00:49:16.124820 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.124836 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:16.124939 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:16.125677 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:16.125680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:16.125696 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:16.125793 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:16.125799 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.126670 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.126675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:16.127828 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:16.127837 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.128560 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.128563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:16.128651 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:16.128674 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:49:16.128774 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 00:49:16.128848 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":1}
I20260806 00:49:16.129732 26734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.124894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:35801 (local address 127.0.0.1:41018)
0806 00:49:16.125081 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.125096 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:16.125184 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:16.126226 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:16.126229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:16.126250 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:16.126337 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:16.126343 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.127273 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.127278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:16.128196 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:16.128207 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.129299 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.129303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:16.129405 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:16.129429 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 00:49:16.129470 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 00:49:16.129605 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":18}
I20260806 00:49:16.129822 26529 tablet_service.cc:1467] Processing CreateTablet for tablet 0d7777faddf643e18c9d0e9d88443e2b (DEFAULT_TABLE table=test-table [id=15ae60fa27914dafa17cc69f14f7e06e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:16.129974 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.125133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41018 (local address 127.15.147.129:35801)
0806 00:49:16.125947 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:16.125952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:16.125969 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:49:16.126036 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:16.126039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:16.126088 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:16.126189 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:16.126478 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.127167 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.128592 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.129177 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.129621 (+ 444us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:16.129671 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:16.129707 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 00:49:16.129790 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":69,"threads_started":1}
I20260806 00:49:16.130169 26529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d7777faddf643e18c9d0e9d88443e2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:16.131235 26761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.125513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:45675 (local address 127.0.0.1:49274)
0806 00:49:16.126227 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.126240 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:16.126324 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:16.126734 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:16.126737 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:16.126754 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:16.126833 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:16.126838 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.127831 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.127835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:16.130068 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:16.130080 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.130804 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.130807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:16.130892 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:16.130912 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:49:16.131009 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 00:49:16.131077 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":33,"thread_start_us":116,"threads_started":1}
I20260806 00:49:16.131251 26764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.125658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49274 (local address 127.15.147.130:45675)
0806 00:49:16.126084 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:16.126089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:16.126418 (+ 329us) server_negotiation.cc:407] Connection header received
0806 00:49:16.126480 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:16.126483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:16.126542 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:16.126622 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:16.126952 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.127713 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.130197 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.130687 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.130815 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:16.130864 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:16.131021 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 00:49:16.131101 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":70,"threads_started":1}
I20260806 00:49:16.131832 26333 tablet_service.cc:1467] Processing CreateTablet for tablet 0d7777faddf643e18c9d0e9d88443e2b (DEFAULT_TABLE table=test-table [id=15ae60fa27914dafa17cc69f14f7e06e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:16.132113 26333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d7777faddf643e18c9d0e9d88443e2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:16.133201 26138 tablet_service.cc:1467] Processing CreateTablet for tablet 0d7777faddf643e18c9d0e9d88443e2b (DEFAULT_TABLE table=test-table [id=15ae60fa27914dafa17cc69f14f7e06e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:16.133492 26138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d7777faddf643e18c9d0e9d88443e2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:16.134438 26766 tablet_bootstrap.cc:492] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c: Bootstrap starting.
I20260806 00:49:16.134774 26765 tablet_bootstrap.cc:492] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0: Bootstrap starting.
I20260806 00:49:16.135418 26766 tablet_bootstrap.cc:654] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:16.135704 26765 tablet_bootstrap.cc:654] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:16.135753 26767 tablet_bootstrap.cc:492] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad: Bootstrap starting.
I20260806 00:49:16.136723 26767 tablet_bootstrap.cc:654] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:16.136930 26766 tablet_bootstrap.cc:492] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c: No bootstrap required, opened a new log
I20260806 00:49:16.136983 26765 tablet_bootstrap.cc:492] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0: No bootstrap required, opened a new log
I20260806 00:49:16.137032 26766 ts_tablet_manager.cc:1397] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:49:16.137074 26765 ts_tablet_manager.cc:1397] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 00:49:16.137480 26765 raft_consensus.cc:348] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } } peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } }
I20260806 00:49:16.137480 26766 raft_consensus.cc:348] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } } peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } }
I20260806 00:49:16.137606 26765 raft_consensus.cc:374] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:16.137606 26766 raft_consensus.cc:374] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:16.137645 26766 raft_consensus.cc:729] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 308ca6c0a0b7403ba8c7187f1fd4f98c, State: Initialized, Role: FOLLOWER
I20260806 00:49:16.137645 26765 raft_consensus.cc:729] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84469497d4f45d6860a4c4603d326d0, State: Initialized, Role: FOLLOWER
I20260806 00:49:16.137768 26766 consensus_queue.cc:260] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } } peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } }
I20260806 00:49:16.137768 26765 consensus_queue.cc:260] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } } peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } }
I20260806 00:49:16.137951 26767 tablet_bootstrap.cc:492] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad: No bootstrap required, opened a new log
I20260806 00:49:16.138054 26767 ts_tablet_manager.cc:1397] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 00:49:16.138132 26765 ts_tablet_manager.cc:1428] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:49:16.138132 26766 ts_tablet_manager.cc:1428] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:49:16.138521 26767 raft_consensus.cc:348] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } } peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } }
I20260806 00:49:16.138646 26767 raft_consensus.cc:374] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:16.138688 26767 raft_consensus.cc:729] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9c22b36dc1d43afbe4f6d3e2d2592ad, State: Initialized, Role: FOLLOWER
I20260806 00:49:16.138871 26767 consensus_queue.cc:260] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } } peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } }
I20260806 00:49:16.139321 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.138657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.130:44381)
0806 00:49:16.139153 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.139214 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":96,"threads_started":1}
I20260806 00:49:16.140156 26778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.139353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.131:35667)
0806 00:49:16.139985 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.140050 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":94,"threads_started":1}
I20260806 00:49:16.140651 26767 ts_tablet_manager.cc:1428] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:49:16.141266 26428 heartbeater.cc:503] Master 127.15.147.189:33281 was elected leader, sending a full tablet report...
I20260806 00:49:16.142795 26237 heartbeater.cc:503] Master 127.15.147.189:33281 was elected leader, sending a full tablet report...
I20260806 00:49:16.143402 26782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.141883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.129:38305)
0806 00:49:16.143231 (+ 1348us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.143287 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1281,"thread_start_us":94,"threads_started":1}
I20260806 00:49:16.144243 26782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.143843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.129:55463)
0806 00:49:16.144121 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.144170 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":110,"threads_started":1}
I20260806 00:49:16.287555 26781 raft_consensus.cc:482] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:49:16.287719 26781 raft_consensus.cc:504] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } } peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } }
I20260806 00:49:16.288918 26781 leader_election.cc:288] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c84469497d4f45d6860a4c4603d326d0 (127.15.147.131:36609), 308ca6c0a0b7403ba8c7187f1fd4f98c (127.15.147.130:45675)
I20260806 00:49:16.292917 26759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.289404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:34333 (local address 127.15.147.131:36609)
0806 00:49:16.289557 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:16.289564 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:16.289588 (+ 24us) server_negotiation.cc:407] Connection header received
0806 00:49:16.289705 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:16.289709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:16.289779 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:16.289875 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:16.290204 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.290719 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.291840 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.292253 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.292434 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:16.292495 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:16.292701 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 00:49:16.292781 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:49:16.292933 26783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.289285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:36609 (local address 127.15.147.129:34333)
0806 00:49:16.289435 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.289453 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:16.289583 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:16.289914 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:16.289917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:16.289938 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:16.290077 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:16.290083 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.290851 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.290855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:16.291712 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:16.291721 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.292438 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.292442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:16.292561 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:16.292580 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 00:49:16.292683 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 00:49:16.292756 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":1}
I20260806 00:49:16.293319 26764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.289787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:37679 (local address 127.15.147.130:45675)
0806 00:49:16.289960 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:16.289967 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:16.289984 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:49:16.290038 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:16.290040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:16.290081 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:16.290164 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:16.290465 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.290974 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.292112 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.292628 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.292772 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:16.292825 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:16.293104 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 00:49:16.293194 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:49:16.293476 26564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d7777faddf643e18c9d0e9d88443e2b" candidate_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84469497d4f45d6860a4c4603d326d0" is_pre_election: true
I20260806 00:49:16.293685 26564 raft_consensus.cc:2454] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9c22b36dc1d43afbe4f6d3e2d2592ad in term 0.
I20260806 00:49:16.293740 26782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.289628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:45675 (local address 127.15.147.129:37679)
0806 00:49:16.289799 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.289812 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:16.289899 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:16.290172 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:16.290175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:16.290191 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:16.290316 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:16.290322 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.291082 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.291087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:16.291944 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:16.291955 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.292791 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.292795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:16.292918 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:16.292937 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 00:49:16.293508 (+ 571us) client_negotiation.cc:240] Negotiation successful
0806 00:49:16.293607 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":60}
I20260806 00:49:16.294198 26074 leader_election.cc:302] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c84469497d4f45d6860a4c4603d326d0, e9c22b36dc1d43afbe4f6d3e2d2592ad; no voters:
I20260806 00:49:16.294469 26781 raft_consensus.cc:2781] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:49:16.294544 26781 raft_consensus.cc:482] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:49:16.294605 26781 raft_consensus.cc:3037] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:16.294924 26360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d7777faddf643e18c9d0e9d88443e2b" candidate_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" is_pre_election: true
I20260806 00:49:16.295162 26360 raft_consensus.cc:2454] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9c22b36dc1d43afbe4f6d3e2d2592ad in term 0.
I20260806 00:49:16.295939 26781 raft_consensus.cc:504] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } } peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } }
I20260806 00:49:16.296348 26781 leader_election.cc:288] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [CANDIDATE]: Term 1 election: Requested vote from peers c84469497d4f45d6860a4c4603d326d0 (127.15.147.131:36609), 308ca6c0a0b7403ba8c7187f1fd4f98c (127.15.147.130:45675)
I20260806 00:49:16.296669 26564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d7777faddf643e18c9d0e9d88443e2b" candidate_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84469497d4f45d6860a4c4603d326d0"
I20260806 00:49:16.296737 26360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d7777faddf643e18c9d0e9d88443e2b" candidate_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c"
I20260806 00:49:16.296846 26360 raft_consensus.cc:3037] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:16.296846 26564 raft_consensus.cc:3037] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:16.298100 26564 raft_consensus.cc:2454] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9c22b36dc1d43afbe4f6d3e2d2592ad in term 1.
I20260806 00:49:16.298146 26360 raft_consensus.cc:2454] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9c22b36dc1d43afbe4f6d3e2d2592ad in term 1.
I20260806 00:49:16.298431 26074 leader_election.cc:302] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c84469497d4f45d6860a4c4603d326d0, e9c22b36dc1d43afbe4f6d3e2d2592ad; no voters:
I20260806 00:49:16.298691 26781 raft_consensus.cc:2781] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:16.298947 26781 raft_consensus.cc:686] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 1 LEADER]: Becoming Leader. State: Replica: e9c22b36dc1d43afbe4f6d3e2d2592ad, State: Running, Role: LEADER
I20260806 00:49:16.299360 26781 consensus_queue.cc:237] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } } peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } }
I20260806 00:49:16.299672 26782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.299409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.129:57281)
0806 00:49:16.299551 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.299608 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 00:49:16.302018 25725 catalog_manager.cc:5495] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad reported cstate change: term changed from 0 to 1, leader changed from <none> to e9c22b36dc1d43afbe4f6d3e2d2592ad (127.15.147.129). New cstate: current_term: 1 leader_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9c22b36dc1d43afbe4f6d3e2d2592ad" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 } health_report { overall_health: UNKNOWN } } }
I20260806 00:49:16.321245 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.317868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41032 (local address 127.15.147.129:35801)
0806 00:49:16.317986 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:16.317990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:16.318008 (+ 18us) server_negotiation.cc:407] Connection header received
0806 00:49:16.318076 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:16.318078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:16.318129 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:16.318213 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:49:16.318469 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.318945 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.319782 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:16.319989 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:16.320075 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:16.320298 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:49:16.320865 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:49:16.320969 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:16.321073 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 00:49:16.321142 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 00:49:16.324069 26564 raft_consensus.cc:1261] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0 [term 1 FOLLOWER]: Refusing update from remote peer e9c22b36dc1d43afbe4f6d3e2d2592ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:16.324465 26360 raft_consensus.cc:1261] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 1 FOLLOWER]: Refusing update from remote peer e9c22b36dc1d43afbe4f6d3e2d2592ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:16.324662 26785 consensus_queue.cc:1035] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "c84469497d4f45d6860a4c4603d326d0" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 36609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:16.325012 26781 consensus_queue.cc:1035] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "308ca6c0a0b7403ba8c7187f1fd4f98c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 45675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:16.325846 26639 heartbeater.cc:503] Master 127.15.147.189:33281 was elected leader, sending a full tablet report...
I20260806 00:49:16.326565 26778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.326264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.131:35671)
0806 00:49:16.326393 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.326467 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 00:49:16.327054 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.326747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.130:42123)
0806 00:49:16.326933 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.326986 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260806 00:49:16.330770 26778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.330454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.131:39985)
0806 00:49:16.330625 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.330684 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260806 00:49:16.332049 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.331765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.130:33817)
0806 00:49:16.331921 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.331976 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 00:49:16.332525 26782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.332214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.129:46653)
0806 00:49:16.332397 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.332452 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260806 00:49:16.337423 26782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.337139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:35655 (local address 127.15.147.129:43401)
0806 00:49:16.337267 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.337330 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:35655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 00:49:16.338789 25724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:49:16.338937 25724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:49:16.343079 25724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a526db9ea2b4fdcad2f43cd537a87cd: Sending DeleteTablet for 3 replicas of tablet 0d7777faddf643e18c9d0e9d88443e2b
I20260806 00:49:16.343758 26529 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d7777faddf643e18c9d0e9d88443e2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:16 UTC) from {username='oryx', principal='oryx/127.15.147.189@KRBTEST.COM'} at 127.0.0.1:54558
I20260806 00:49:16.343840 26138 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d7777faddf643e18c9d0e9d88443e2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:16 UTC) from {username='oryx', principal='oryx/127.15.147.189@KRBTEST.COM'} at 127.0.0.1:41018
I20260806 00:49:16.343994 26333 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d7777faddf643e18c9d0e9d88443e2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:16 UTC) from {username='oryx', principal='oryx/127.15.147.189@KRBTEST.COM'} at 127.0.0.1:49274
I20260806 00:49:16.344422 26808 tablet_replica.cc:330] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c: stopping tablet replica
I20260806 00:49:16.344547 26807 tablet_replica.cc:330] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad: stopping tablet replica
I20260806 00:49:16.344662 26808 raft_consensus.cc:2229] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:16.344686 26807 raft_consensus.cc:2229] T 0d7777faddf643e18c9d0e9d88443e2b P e9c22b36dc1d43afbe4f6d3e2d2592ad [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:49:16.344889 26808 raft_consensus.cc:2258] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:16.344918 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 25931
I20260806 00:49:16.348088 26805 tablet_replica.cc:330] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0: stopping tablet replica
I20260806 00:49:16.348254 26805 raft_consensus.cc:2229] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:16.348441 26805 raft_consensus.cc:2258] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:16.348657 26808 ts_tablet_manager.cc:1905] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:16.351526 26808 ts_tablet_manager.cc:1918] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:49:16.351631 26808 log.cc:1196] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/0d7777faddf643e18c9d0e9d88443e2b
I20260806 00:49:16.351964 26808 ts_tablet_manager.cc:1939] T 0d7777faddf643e18c9d0e9d88443e2b P 308ca6c0a0b7403ba8c7187f1fd4f98c: Deleting consensus metadata
I20260806 00:49:16.352288 26805 ts_tablet_manager.cc:1905] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:16.352741 25708 catalog_manager.cc:4841] TS 308ca6c0a0b7403ba8c7187f1fd4f98c (127.15.147.130:45675): tablet 0d7777faddf643e18c9d0e9d88443e2b (table test-table [id=15ae60fa27914dafa17cc69f14f7e06e]) successfully deleted
I20260806 00:49:16.354519 26805 ts_tablet_manager.cc:1918] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:49:16.354588 26805 log.cc:1196] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/0d7777faddf643e18c9d0e9d88443e2b
I20260806 00:49:16.354911 26805 ts_tablet_manager.cc:1939] T 0d7777faddf643e18c9d0e9d88443e2b P c84469497d4f45d6860a4c4603d326d0: Deleting consensus metadata
I20260806 00:49:16.355628 25708 catalog_manager.cc:4841] TS c84469497d4f45d6860a4c4603d326d0 (127.15.147.131:36609): tablet 0d7777faddf643e18c9d0e9d88443e2b (table test-table [id=15ae60fa27914dafa17cc69f14f7e06e]) successfully deleted
I20260806 00:49:16.358263 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 26246
I20260806 00:49:16.358340 26761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:16.358090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:35801 (local address 127.0.0.1:41046)
0806 00:49:16.358215 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:16.358270 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:35801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 00:49:16.361776 25706 catalog_manager.cc:4584] TS e9c22b36dc1d43afbe4f6d3e2d2592ad (127.15.147.129:35801): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0d7777faddf643e18c9d0e9d88443e2b: Network error: Client connection negotiation failed: client connection to 127.15.147.129:35801: connect: Connection refused (error 111)
I20260806 00:49:16.373376 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 26434
I20260806 00:49:16.386348 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 25538
I20260806 00:49:16.398248 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 25691
2026-08-06T00:49:16Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5932 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:49:16 dist-test-slave-45dx krb5kdc[26811](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:49:16 dist-test-slave-45dx krb5kdc[26811](info): set up 2 sockets
Aug 06 00:49:16 dist-test-slave-45dx krb5kdc[26811](info): commencing operation
krb5kdc: starting...
W20260806 00:49:17.619539 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:49:17 dist-test-slave-45dx krb5kdc[26811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:49:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:49:17Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.190.keytab.
Entry for principal oryx/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.190.keytab.
I20260806 00:49:17.749927 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:41237
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.190.keytab
--principal=oryx/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:39177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:41237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:17.870627 26829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:17.870944 26829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:17.871022 26829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:17.875128 26829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:49:17.875216 26829 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 00:49:17.875247 26829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:17.875295 26829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:17.875347 26829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:49:17.875383 26829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:49:17.880852 26829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:39177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:41237
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.190.keytab
--principal=oryx/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:41237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:17.882006 26829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:17.883091 26829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:17.891875 26840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:17.891865 26842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:17.892030 26829 server_base.cc:1034] running on GCE node
W20260806 00:49:17.892218 26839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:17.892896 26829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:17.894032 26829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:17.895210 26829 hybrid_clock.cc:648] HybridClock initialized: now 1785977357895184 us; error 39 us; skew 500 ppm
Aug 06 00:49:17 dist-test-slave-45dx krb5kdc[26811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977357, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:17.898334 26829 init.cc:377] Logged in from keytab as oryx/127.15.147.190@KRBTEST.COM (short username oryx)
I20260806 00:49:17.899341 26829 webserver.cc:460] Webserver started at http://127.15.147.190:35391/ using document root <none> and password file <none>
I20260806 00:49:17.899919 26829 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:17.899982 26829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:17.900161 26829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:17.902403 26829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "573d75e7bcd943bf900f0b7313c8bd35"
format_stamp: "Formatted at 2026-08-06 00:49:17 on dist-test-slave-45dx"
server_key: "c653fba25f35c56ffee831becbcf6dcb"
server_key_iv: "a0a5a180ed25e62b93446c0faedf2062"
server_key_version: "encryptionkey@0"
I20260806 00:49:17.902964 26829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "573d75e7bcd943bf900f0b7313c8bd35"
format_stamp: "Formatted at 2026-08-06 00:49:17 on dist-test-slave-45dx"
server_key: "c653fba25f35c56ffee831becbcf6dcb"
server_key_iv: "a0a5a180ed25e62b93446c0faedf2062"
server_key_version: "encryptionkey@0"
I20260806 00:49:17.906957 26829 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:49:17.909533 26849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:17.910782 26829 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:49:17.910912 26829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "573d75e7bcd943bf900f0b7313c8bd35"
format_stamp: "Formatted at 2026-08-06 00:49:17 on dist-test-slave-45dx"
server_key: "c653fba25f35c56ffee831becbcf6dcb"
server_key_iv: "a0a5a180ed25e62b93446c0faedf2062"
server_key_version: "encryptionkey@0"
I20260806 00:49:17.911006 26829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:17.932492 26829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:17.935926 26829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:17.936100 26829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:17.944509 26829 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:41237
I20260806 00:49:17.944518 26936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:41237 every 8 connection(s)
I20260806 00:49:17.945468 26829 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:49:17.947697 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 26829
I20260806 00:49:17.947829 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:49:17.948135 15950 external_mini_cluster.cc:1428] Setting key ec79d188751fef45d4c21b94e1e547e1
I20260806 00:49:17.948523 26937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:17.955780 26937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35: Bootstrap starting.
Aug 06 00:49:17 dist-test-slave-45dx krb5kdc[26811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM
I20260806 00:49:17.958591 26937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:17.959966 26937 log.cc:824] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:17.961944 26937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35: No bootstrap required, opened a new log
I20260806 00:49:17.963007 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:17.949544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44790 (local address 127.15.147.190:41237)
0806 00:49:17.949971 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:17.949991 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:17.950035 (+ 44us) server_negotiation.cc:407] Connection header received
0806 00:49:17.950715 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:17.950732 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:17.951050 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:17.951326 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:17.952293 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:17.953396 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:17.954605 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:17.955068 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:17.955205 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:17.957983 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:17.957990 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:17.958014 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:17.958046 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:17.960129 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:17.960598 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:17.960604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:17.960609 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:17.960729 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:17.961011 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:17.961014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:17.961016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:17.961438 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:17.961640 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:17.961968 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 00:49:17.962286 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":139,"threads_started":1}
I20260806 00:49:17.965430 26937 raft_consensus.cc:348] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573d75e7bcd943bf900f0b7313c8bd35" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41237 } }
I20260806 00:49:17.965721 26937 raft_consensus.cc:374] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:17.965801 26937 raft_consensus.cc:729] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 573d75e7bcd943bf900f0b7313c8bd35, State: Initialized, Role: FOLLOWER
I20260806 00:49:17.966217 26937 consensus_queue.cc:260] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573d75e7bcd943bf900f0b7313c8bd35" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41237 } }
I20260806 00:49:17.966368 26937 raft_consensus.cc:388] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:49:17.966452 26937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:49:17.966559 26937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:17.967654 26937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573d75e7bcd943bf900f0b7313c8bd35" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41237 } }
I20260806 00:49:17.967983 26937 leader_election.cc:302] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 573d75e7bcd943bf900f0b7313c8bd35; no voters:
I20260806 00:49:17.968348 26937 leader_election.cc:288] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:49:17.968457 26943 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:17.968681 26943 raft_consensus.cc:686] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35 [term 1 LEADER]: Becoming Leader. State: Replica: 573d75e7bcd943bf900f0b7313c8bd35, State: Running, Role: LEADER
I20260806 00:49:17.968958 26943 consensus_queue.cc:237] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573d75e7bcd943bf900f0b7313c8bd35" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41237 } }
I20260806 00:49:17.969543 26937 sys_catalog.cc:564] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:49:17.970705 26944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "573d75e7bcd943bf900f0b7313c8bd35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573d75e7bcd943bf900f0b7313c8bd35" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41237 } } }
I20260806 00:49:17.970705 26945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 573d75e7bcd943bf900f0b7313c8bd35. Latest consensus state: current_term: 1 leader_uuid: "573d75e7bcd943bf900f0b7313c8bd35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573d75e7bcd943bf900f0b7313c8bd35" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41237 } } }
I20260806 00:49:17.970866 26944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:17.970870 26945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:17.971312 26953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:49:17.974274 26953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:49:17.979673 26953 catalog_manager.cc:1347] Generated new cluster ID: f1a60d2fdddc4cd6a0b2c50663d81e13
I20260806 00:49:17.979759 26953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:49:17.985926 26953 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:49:17.987217 26953 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:49:17.996632 26953 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35: Generated new TSK 0
I20260806 00:49:17.997499 26953 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.129.keytab.
Entry for principal oryx/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.129.keytab.
I20260806 00:49:18.037236 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:41237
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.129.keytab
--principal=oryx/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:39177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:18.153234 26974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:18.153486 26974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:18.153529 26974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:18.157506 26974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:18.157557 26974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:18.157593 26974 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:49:18.157667 26974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:49:18.162544 26974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:39177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.129.keytab
--principal=oryx/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.147.190:41237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:18.163609 26974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:18.164662 26974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:18.174640 26983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:18.174626 26984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:18.174734 26986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:18.175004 26974 server_base.cc:1034] running on GCE node
I20260806 00:49:18.175431 26974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:18.176061 26974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:18.177280 26974 hybrid_clock.cc:648] HybridClock initialized: now 1785977358177266 us; error 34 us; skew 500 ppm
Aug 06 00:49:18 dist-test-slave-45dx krb5kdc[26811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977358, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:18.180110 26974 init.cc:377] Logged in from keytab as oryx/127.15.147.129@KRBTEST.COM (short username oryx)
I20260806 00:49:18.181072 26974 webserver.cc:460] Webserver started at http://127.15.147.129:44785/ using document root <none> and password file <none>
I20260806 00:49:18.181639 26974 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:18.181731 26974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:18.181981 26974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:18.184142 26974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "6a4ee92563e1402997d02ba8b1fd8676"
format_stamp: "Formatted at 2026-08-06 00:49:18 on dist-test-slave-45dx"
server_key: "cc2d3c7ea4bbcafc203330e03fee5793"
server_key_iv: "3bad326393847885ed5dd194b5e67983"
server_key_version: "encryptionkey@0"
I20260806 00:49:18.184818 26974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "6a4ee92563e1402997d02ba8b1fd8676"
format_stamp: "Formatted at 2026-08-06 00:49:18 on dist-test-slave-45dx"
server_key: "cc2d3c7ea4bbcafc203330e03fee5793"
server_key_iv: "3bad326393847885ed5dd194b5e67983"
server_key_version: "encryptionkey@0"
I20260806 00:49:18.188836 26974 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:49:18.191241 26997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:18.192512 26974 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:49:18.192637 26974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "6a4ee92563e1402997d02ba8b1fd8676"
format_stamp: "Formatted at 2026-08-06 00:49:18 on dist-test-slave-45dx"
server_key: "cc2d3c7ea4bbcafc203330e03fee5793"
server_key_iv: "3bad326393847885ed5dd194b5e67983"
server_key_version: "encryptionkey@0"
I20260806 00:49:18.192749 26974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:18.209708 26974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:18.212635 26974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:18.212839 26974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:18.214367 26974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:18.214447 26974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:18.214532 26974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:18.214586 26974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:49:18 dist-test-slave-45dx krb5kdc[26811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977358, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM
I20260806 00:49:18.225911 26974 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:33443
I20260806 00:49:18.226293 27157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:33443 every 8 connection(s)
I20260806 00:49:18.226956 26974 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:49:18.229137 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.215280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:53637 (local address 127.15.147.190:41237)
0806 00:49:18.215415 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:18.215419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:18.216600 (+ 1181us) server_negotiation.cc:407] Connection header received
0806 00:49:18.217719 (+ 1119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:18.217723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:18.217805 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:18.217868 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:18.219785 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.220744 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.222049 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.222292 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.222412 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.225236 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:18.225239 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:18.225241 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:18.225268 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:18.226957 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:18.227573 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:18.227576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:18.227578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:18.227671 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:18.228034 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:18.228036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:18.228037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:18.228218 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:18.228337 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:18.228894 (+ 557us) server_negotiation.cc:299] Negotiation successful
0806 00:49:18.229006 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 00:49:18.230007 27012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.215909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:41237 (local address 127.15.147.129:53637)
0806 00:49:18.216451 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:18.216497 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:18.217458 (+ 961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:18.218321 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:18.218335 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:18.218816 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:18.219565 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.219584 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.220891 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.220895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.221921 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.221932 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.222460 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.222464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.222625 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.223244 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:18.223260 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:18.225070 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:18.227098 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:18.227104 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:18.227118 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:18.227451 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:18.227761 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:18.227766 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:18.227768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:18.227937 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:18.228405 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:18.228412 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:18.228681 (+ 269us) client_negotiation.cc:769] Sending connection context
0806 00:49:18.228854 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 00:49:18.229104 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":143,"threads_started":1}
I20260806 00:49:18.231324 27158 heartbeater.cc:347] Connected to a master server at 127.15.147.190:41237
I20260806 00:49:18.231487 27158 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:18.232048 27158 heartbeater.cc:511] Master 127.15.147.190:41237 requested a full tablet report, sending...
I20260806 00:49:18.233645 26867 ts_manager.cc:194] Registered new tserver with Master: 6a4ee92563e1402997d02ba8b1fd8676 (127.15.147.129:33443)
I20260806 00:49:18.234457 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 26974
I20260806 00:49:18.234578 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:49:18.234862 15950 external_mini_cluster.cc:1428] Setting key e60716548e91e0d60a191aca15c47db9
I20260806 00:49:18.234917 26867 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:53637
I20260806 00:49:18.244025 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.233407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:41237 (local address 127.15.147.129:51661)
0806 00:49:18.235631 (+ 2224us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:18.235650 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:18.235775 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:18.236595 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:18.236599 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:18.236839 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:18.237003 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.237014 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.237864 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.237871 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.238825 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.238836 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.239476 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.239478 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.239550 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.240129 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:18.240154 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:18.240542 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:18.242523 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:18.242525 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:18.242526 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:18.242711 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:18.243012 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:18.243014 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:18.243016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:18.243088 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:18.243599 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:18.243601 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:18.243674 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 00:49:18.243749 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 00:49:18.243849 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2119,"thread_start_us":92,"threads_started":1}
I20260806 00:49:18.244237 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.233509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:51661 (local address 127.15.147.190:41237)
0806 00:49:18.233662 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:18.233668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:18.235734 (+ 2066us) server_negotiation.cc:407] Connection header received
0806 00:49:18.236417 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:18.236421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:18.236468 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:18.236554 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:18.237146 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.237727 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.238950 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.239344 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.239467 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.240673 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:18.240679 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:18.240684 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:18.240716 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:18.242405 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:18.242805 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:18.242812 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:18.242817 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:18.242916 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:18.243188 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:18.243194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:18.243214 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:18.243490 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:18.243621 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:18.243840 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 00:49:18.243982 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
WARNING: no policy specified for oryx/127.15.147.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.147.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.130.keytab.
Entry for principal oryx/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.130.keytab.
I20260806 00:49:18.267619 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:41237
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.130.keytab
--principal=oryx/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:39177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:18.382393 27171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:18.382679 27171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:18.382755 27171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:18.386807 27171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:18.386909 27171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:18.386981 27171 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:49:18.387059 27171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:49:18.392547 27171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:39177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.130.keytab
--principal=oryx/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.147.190:41237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:18.393765 27171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:18.394896 27171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:18.405097 27183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:18.405114 27181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:18.405164 27171 server_base.cc:1034] running on GCE node
W20260806 00:49:18.405105 27185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:18.405699 27171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:18.406286 27171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:18.407452 27171 hybrid_clock.cc:648] HybridClock initialized: now 1785977358407433 us; error 29 us; skew 500 ppm
Aug 06 00:49:18 dist-test-slave-45dx krb5kdc[26811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977358, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:18.410259 27171 init.cc:377] Logged in from keytab as oryx/127.15.147.130@KRBTEST.COM (short username oryx)
I20260806 00:49:18.411082 27171 webserver.cc:460] Webserver started at http://127.15.147.130:33087/ using document root <none> and password file <none>
I20260806 00:49:18.411645 27171 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:18.411707 27171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:18.411942 27171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:18.414017 27171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "f6f3901094e34f41a3d4bdc5436cc8b5"
format_stamp: "Formatted at 2026-08-06 00:49:18 on dist-test-slave-45dx"
server_key: "36e57522f18c6029491b71525057aa2b"
server_key_iv: "bd378fb0c3542436425dfee97e6e1d3b"
server_key_version: "encryptionkey@0"
I20260806 00:49:18.414667 27171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "f6f3901094e34f41a3d4bdc5436cc8b5"
format_stamp: "Formatted at 2026-08-06 00:49:18 on dist-test-slave-45dx"
server_key: "36e57522f18c6029491b71525057aa2b"
server_key_iv: "bd378fb0c3542436425dfee97e6e1d3b"
server_key_version: "encryptionkey@0"
I20260806 00:49:18.418432 27171 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 00:49:18.420810 27194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:18.422013 27171 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:49:18.422132 27171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "f6f3901094e34f41a3d4bdc5436cc8b5"
format_stamp: "Formatted at 2026-08-06 00:49:18 on dist-test-slave-45dx"
server_key: "36e57522f18c6029491b71525057aa2b"
server_key_iv: "bd378fb0c3542436425dfee97e6e1d3b"
server_key_version: "encryptionkey@0"
I20260806 00:49:18.422235 27171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:18.443951 27171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:18.446944 27171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:18.447161 27171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:18.448756 27171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:18.448807 27171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:18.448848 27171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:18.448895 27171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:49:18 dist-test-slave-45dx krb5kdc[26811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977358, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM
I20260806 00:49:18.460249 27171 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:39021
I20260806 00:49:18.460350 27357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:39021 every 8 connection(s)
I20260806 00:49:18.461269 27171 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:49:18.463356 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.449515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:40331 (local address 127.15.147.190:41237)
0806 00:49:18.449665 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:18.449671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:18.450608 (+ 937us) server_negotiation.cc:407] Connection header received
0806 00:49:18.451617 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:18.451622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:18.451684 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:18.451750 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:18.453305 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.453915 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.455160 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.455434 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.455553 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.459526 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:18.459530 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:18.459533 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:18.459562 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:18.461259 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:18.461852 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:18.461855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:18.461857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:18.461955 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:18.462346 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:18.462350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:18.462351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:18.462540 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:18.462628 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:18.463121 (+ 493us) server_negotiation.cc:299] Negotiation successful
0806 00:49:18.463229 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:49:18.464251 27205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.449949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:41237 (local address 127.15.147.130:40331)
0806 00:49:18.450429 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:18.450480 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:18.451354 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:18.452170 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:18.452176 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:18.452611 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:18.453103 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.453118 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.454057 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.454060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.455027 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.455038 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.455600 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.455604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.455835 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.456519 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:18.456546 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:18.459350 (+ 2804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:18.461387 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:18.461393 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:18.461407 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:18.461690 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:18.462067 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:18.462071 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:18.462073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:18.462239 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:18.462655 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:18.462665 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:18.462908 (+ 243us) client_negotiation.cc:769] Sending connection context
0806 00:49:18.463068 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 00:49:18.463319 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":129,"threads_started":1}
I20260806 00:49:18.464759 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 27171
I20260806 00:49:18.464887 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:49:18.465250 15950 external_mini_cluster.cc:1428] Setting key 1ccf5f08dba64a0363315b787a7d8001
I20260806 00:49:18.465621 27358 heartbeater.cc:347] Connected to a master server at 127.15.147.190:41237
I20260806 00:49:18.465772 27358 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:18.466355 27358 heartbeater.cc:511] Master 127.15.147.190:41237 requested a full tablet report, sending...
I20260806 00:49:18.467516 26867 ts_manager.cc:194] Registered new tserver with Master: f6f3901094e34f41a3d4bdc5436cc8b5 (127.15.147.130:39021)
I20260806 00:49:18.468523 26867 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:40331
WARNING: no policy specified for oryx/127.15.147.131@KRBTEST.COM; defaulting to no policy
I20260806 00:49:18.478673 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.467990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:36453 (local address 127.15.147.190:41237)
0806 00:49:18.468152 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:18.468157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:18.468733 (+ 576us) server_negotiation.cc:407] Connection header received
0806 00:49:18.468887 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:18.468891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:18.468942 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:18.469033 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:18.471487 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.472056 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.473155 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.473415 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.473533 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.474834 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:18.474840 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:18.474843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:18.474870 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:18.476850 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:18.477317 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:18.477321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:18.477323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:18.477418 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:18.477749 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:18.477753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:18.477755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:18.477989 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:18.478112 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:18.478450 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 00:49:18.478554 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 00:49:18.478675 27366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.468151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:41237 (local address 127.15.147.130:36453)
0806 00:49:18.468621 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:18.468637 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:18.468780 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:18.469053 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:18.469056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:18.471221 (+ 2165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:18.471349 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.471357 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.472209 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.472213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.473016 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.473027 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.473541 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.473545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.473652 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.474262 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:18.474276 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:18.474661 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:18.476977 (+ 2316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:18.476980 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:18.476981 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:18.477194 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:18.477531 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:18.477534 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:18.477536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:18.477648 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:18.478101 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:18.478104 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:18.478211 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 00:49:18.478360 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 00:49:18.478521 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":237,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.147.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.131.keytab.
Entry for principal oryx/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.131.keytab.
I20260806 00:49:18.499050 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:41237
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.131.keytab
--principal=oryx/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:39177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:18.617187 27370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:18.617450 27370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:18.617535 27370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:18.621414 27370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:18.621469 27370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:18.621544 27370 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 00:49:18.621621 27370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:49:18.627095 27370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:39177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.131.keytab
--principal=oryx/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.147.190:41237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:18.628166 27370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:18.629258 27370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:18.639261 27382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:18.639268 27376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:18.639304 27370 server_base.cc:1034] running on GCE node
W20260806 00:49:18.639276 27377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:18.639986 27370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:18.640681 27370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:18.641889 27370 hybrid_clock.cc:648] HybridClock initialized: now 1785977358641851 us; error 51 us; skew 500 ppm
Aug 06 00:49:18 dist-test-slave-45dx krb5kdc[26811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977358, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:18.644747 27370 init.cc:377] Logged in from keytab as oryx/127.15.147.131@KRBTEST.COM (short username oryx)
I20260806 00:49:18.645633 27370 webserver.cc:460] Webserver started at http://127.15.147.131:40023/ using document root <none> and password file <none>
I20260806 00:49:18.646201 27370 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:18.646291 27370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:18.646530 27370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:18.648690 27370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "c38adac127f84c8bbdb8a91ea391525e"
format_stamp: "Formatted at 2026-08-06 00:49:18 on dist-test-slave-45dx"
server_key: "01ca310942dd40c71ff522f2e469a106"
server_key_iv: "6b2f806c21f978e882df46efa40524ac"
server_key_version: "encryptionkey@0"
I20260806 00:49:18.649256 27370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "c38adac127f84c8bbdb8a91ea391525e"
format_stamp: "Formatted at 2026-08-06 00:49:18 on dist-test-slave-45dx"
server_key: "01ca310942dd40c71ff522f2e469a106"
server_key_iv: "6b2f806c21f978e882df46efa40524ac"
server_key_version: "encryptionkey@0"
I20260806 00:49:18.653031 27370 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:49:18.655428 27393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:18.656715 27370 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:49:18.656845 27370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "c38adac127f84c8bbdb8a91ea391525e"
format_stamp: "Formatted at 2026-08-06 00:49:18 on dist-test-slave-45dx"
server_key: "01ca310942dd40c71ff522f2e469a106"
server_key_iv: "6b2f806c21f978e882df46efa40524ac"
server_key_version: "encryptionkey@0"
I20260806 00:49:18.656957 27370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:18.675262 27370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:18.678797 27370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:18.679016 27370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:18.680636 27370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:18.680719 27370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:18.680807 27370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:18.680847 27370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 00:49:18 dist-test-slave-45dx krb5kdc[26811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977358, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM
I20260806 00:49:18.692631 27370 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:35525
I20260806 00:49:18.692711 27555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:35525 every 8 connection(s)
I20260806 00:49:18.693675 27370 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:49:18.694940 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.681429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:40661 (local address 127.15.147.190:41237)
0806 00:49:18.681603 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:18.681610 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:18.682471 (+ 861us) server_negotiation.cc:407] Connection header received
0806 00:49:18.683500 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:18.683507 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:18.683576 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:18.683681 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:18.685519 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.686393 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.687622 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.687992 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.688115 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.690944 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:18.690948 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:18.690950 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:18.690985 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:18.692679 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:18.693209 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:18.693213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:18.693214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:18.693315 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:18.693647 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:18.693653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:18.693656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:18.693897 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:18.694002 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:18.694622 (+ 620us) server_negotiation.cc:299] Negotiation successful
0806 00:49:18.694761 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 00:49:18.695703 27403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.681833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:41237 (local address 127.15.147.131:40661)
0806 00:49:18.682299 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:18.682351 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:18.683236 (+ 885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:18.684039 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:18.684049 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:18.684601 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:18.685329 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.685353 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.686607 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.686612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.687482 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.687493 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.688163 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.688168 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.688416 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.689098 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:18.689120 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:18.690774 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:18.692818 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:18.692823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:18.692838 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:18.693110 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:18.693415 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:18.693417 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:18.693418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:18.693551 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:18.694008 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:18.694012 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:18.694343 (+ 331us) client_negotiation.cc:769] Sending connection context
0806 00:49:18.694547 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 00:49:18.694825 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":130,"threads_started":1}
I20260806 00:49:18.696878 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 27370
I20260806 00:49:18.697017 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:49:18.697088 27556 heartbeater.cc:347] Connected to a master server at 127.15.147.190:41237
I20260806 00:49:18.697253 27556 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:18.697407 15950 external_mini_cluster.cc:1428] Setting key 2be01b2368f76aed35df08d8ce438b2c
I20260806 00:49:18.697852 27556 heartbeater.cc:511] Master 127.15.147.190:41237 requested a full tablet report, sending...
I20260806 00:49:18.699010 26870 ts_manager.cc:194] Registered new tserver with Master: c38adac127f84c8bbdb8a91ea391525e (127.15.147.131:35525)
I20260806 00:49:18.699721 26870 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:40661
I20260806 00:49:18.705889 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.699311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:45631 (local address 127.15.147.190:41237)
0806 00:49:18.699421 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:18.699425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:18.699676 (+ 251us) server_negotiation.cc:407] Connection header received
0806 00:49:18.699785 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:18.699788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:18.699850 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:18.699940 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:18.700398 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.700877 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.701682 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.701922 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.702028 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.703036 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:18.703039 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:18.703046 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:18.703068 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:18.704210 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:18.704685 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:18.704689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:18.704691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:18.704771 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:18.705052 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:18.705056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:18.705058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:18.705228 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:18.705335 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:18.705612 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 00:49:18.705724 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:49:18.705911 27564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.699263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:41237 (local address 127.15.147.131:45631)
0806 00:49:18.699603 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:18.699619 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:18.699712 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:18.699956 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:18.699959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:18.700151 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:18.700247 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.700252 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.701005 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.701008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.701561 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.701569 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.702040 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.702044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.702145 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.702580 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:18.702592 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:18.702894 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:18.704350 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:18.704353 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:18.704356 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:18.704578 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:18.704864 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:18.704867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:18.704870 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:18.704958 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:18.705326 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:18.705329 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:18.705430 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 00:49:18.705554 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 00:49:18.705711 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":94,"threads_started":1}
I20260806 00:49:18.711143 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 00:49:18 dist-test-slave-45dx krb5kdc[26811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977358, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 00:49:18 dist-test-slave-45dx krb5kdc[26811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:18 dist-test-slave-45dx krb5kdc[26811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 00:49:18.738009 26940 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:18.731737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44798 (local address 127.15.147.190:41237)
0806 00:49:18.731874 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:18.731878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:18.731891 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:49:18.731936 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:18.731938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:18.731979 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:18.732060 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:18.732920 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.733440 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.734275 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.734487 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.734563 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.737853 (+ 3290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44798: BlockingRecv error: recv got EOF from 127.0.0.1:44798 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 00:49:18 dist-test-slave-45dx krb5kdc[26811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977358, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM
I20260806 00:49:18.747300 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.739514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44806 (local address 127.15.147.190:41237)
0806 00:49:18.739639 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:18.739643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:18.739792 (+ 149us) server_negotiation.cc:407] Connection header received
0806 00:49:18.739882 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:18.739884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:18.739922 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:18.739984 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:18.740622 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.741109 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.741826 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.742036 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.742109 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.744541 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:18.744544 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:18.744546 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:18.744568 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:18.745810 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:18.746183 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:18.746186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:18.746187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:18.746263 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:18.746499 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:18.746502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:18.746504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:18.746671 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:18.746775 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:18.746973 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 00:49:18.747101 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:49:18.750211 26870 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:49:18.753223 26870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:49:18.769101 27587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.762223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:39021 (local address 127.0.0.1:55456)
0806 00:49:18.762595 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:18.762657 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:18.762796 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:18.764144 (+ 1348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:18.764150 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:18.764172 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:18.764380 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.764389 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.765923 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.765927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.767400 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.767412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.768346 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.768349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.768475 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.768529 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 00:49:18.768643 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 00:49:18.768784 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":90,"thread_start_us":94,"threads_started":1}
I20260806 00:49:18.769147 27588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.762395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55456 (local address 127.15.147.130:39021)
0806 00:49:18.763733 (+ 1338us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:18.763742 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:18.763767 (+ 25us) server_negotiation.cc:407] Connection header received
0806 00:49:18.763838 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:18.763841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:18.763986 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:18.764154 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:18.764496 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.765710 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.767539 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.768195 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.768388 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.768531 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:18.768704 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 00:49:18.768815 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1249,"thread_start_us":123,"threads_started":1}
I20260806 00:49:18.769922 27592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.762952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:33443 (local address 127.0.0.1:57214)
0806 00:49:18.763438 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:18.763449 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:18.763514 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:18.764343 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:18.764346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:18.764376 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:18.764598 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.764603 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.766868 (+ 2265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.766872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.767971 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.767982 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.769518 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.769522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.769626 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.769649 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:49:18.769709 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 00:49:18.769772 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":55,"spinlock_wait_cycles":6784,"thread_start_us":98,"threads_started":1}
I20260806 00:49:18.770506 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.763391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57214 (local address 127.15.147.129:33443)
0806 00:49:18.763677 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:18.763685 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:18.763710 (+ 25us) server_negotiation.cc:407] Connection header received
0806 00:49:18.763793 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:18.763800 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:18.764016 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:18.764160 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:18.765985 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.766754 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.768118 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.768979 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.769108 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.770213 (+ 1105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:18.770263 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 00:49:18.770351 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":104,"threads_started":1}
I20260806 00:49:18.771096 27590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.762567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:35525 (local address 127.0.0.1:39850)
0806 00:49:18.763173 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:18.763189 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:18.763298 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:18.764343 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:18.764346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:18.764392 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:18.764596 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.764603 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.765924 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.765927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.769348 (+ 3421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.769359 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.770424 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.770427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.770536 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.770758 (+ 222us) client_negotiation.cc:769] Sending connection context
0806 00:49:18.770850 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 00:49:18.770928 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":1310,"spinlock_wait_cycles":42240,"thread_start_us":121,"threads_started":1}
I20260806 00:49:18.771128 27593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.763037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39850 (local address 127.15.147.131:35525)
0806 00:49:18.763772 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:18.763786 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:18.763803 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:49:18.763865 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:18.763869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:18.764007 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:18.764160 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:18.764731 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.765836 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.769489 (+ 3653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.770298 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.770447 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.770566 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:18.770881 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 00:49:18.770980 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":138,"threads_started":1}
I20260806 00:49:18.772990 27258 tablet_service.cc:1467] Processing CreateTablet for tablet 405c473e8aa147719d81a3ff19937940 (DEFAULT_TABLE table=test-table [id=4bc62d710e6c4e8fbd96190e85473f20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:18.772986 27066 tablet_service.cc:1467] Processing CreateTablet for tablet 405c473e8aa147719d81a3ff19937940 (DEFAULT_TABLE table=test-table [id=4bc62d710e6c4e8fbd96190e85473f20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:18.774006 27066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 405c473e8aa147719d81a3ff19937940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:18.774013 27258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 405c473e8aa147719d81a3ff19937940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:18.779132 27462 tablet_service.cc:1467] Processing CreateTablet for tablet 405c473e8aa147719d81a3ff19937940 (DEFAULT_TABLE table=test-table [id=4bc62d710e6c4e8fbd96190e85473f20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:18.779987 27596 tablet_bootstrap.cc:492] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676: Bootstrap starting.
I20260806 00:49:18.780232 27462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 405c473e8aa147719d81a3ff19937940. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:18.781725 27602 tablet_bootstrap.cc:492] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5: Bootstrap starting.
I20260806 00:49:18.782274 27596 tablet_bootstrap.cc:654] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:18.783174 27596 log.cc:824] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:18.783968 27602 tablet_bootstrap.cc:654] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:18.784909 27602 log.cc:824] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:18.785104 27596 tablet_bootstrap.cc:492] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676: No bootstrap required, opened a new log
I20260806 00:49:18.785336 27596 ts_tablet_manager.cc:1397] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:49:18.787035 27602 tablet_bootstrap.cc:492] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5: No bootstrap required, opened a new log
I20260806 00:49:18.787218 27602 ts_tablet_manager.cc:1397] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:49:18.788231 27610 tablet_bootstrap.cc:492] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e: Bootstrap starting.
I20260806 00:49:18.789357 27596 raft_consensus.cc:348] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:18.789659 27596 raft_consensus.cc:374] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:18.789711 27596 raft_consensus.cc:729] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4ee92563e1402997d02ba8b1fd8676, State: Initialized, Role: FOLLOWER
I20260806 00:49:18.790194 27596 consensus_queue.cc:260] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:18.790673 27610 tablet_bootstrap.cc:654] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:18.790858 27158 heartbeater.cc:503] Master 127.15.147.190:41237 was elected leader, sending a full tablet report...
I20260806 00:49:18.790798 27602 raft_consensus.cc:348] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:18.791051 27602 raft_consensus.cc:374] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:18.791113 27602 raft_consensus.cc:729] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6f3901094e34f41a3d4bdc5436cc8b5, State: Initialized, Role: FOLLOWER
I20260806 00:49:18.791239 27596 ts_tablet_manager.cc:1428] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:49:18.791533 27602 consensus_queue.cc:260] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:18.791888 27610 log.cc:824] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:18.792171 27358 heartbeater.cc:503] Master 127.15.147.190:41237 was elected leader, sending a full tablet report...
I20260806 00:49:18.792505 27602 ts_tablet_manager.cc:1428] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5: Time spent starting tablet: real 0.005s user 0.002s sys 0.005s
I20260806 00:49:18.793936 27610 tablet_bootstrap.cc:492] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e: No bootstrap required, opened a new log
I20260806 00:49:18.794118 27610 ts_tablet_manager.cc:1397] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 00:49:18.797814 27610 raft_consensus.cc:348] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:18.798025 27610 raft_consensus.cc:374] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:18.798074 27610 raft_consensus.cc:729] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c38adac127f84c8bbdb8a91ea391525e, State: Initialized, Role: FOLLOWER
I20260806 00:49:18.798461 27610 consensus_queue.cc:260] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:18.799106 27556 heartbeater.cc:503] Master 127.15.147.190:41237 was elected leader, sending a full tablet report...
I20260806 00:49:18.799388 27610 ts_tablet_manager.cc:1428] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
W20260806 00:49:18.944761 27559 tablet.cc:2369] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:49:18.949261 27612 raft_consensus.cc:482] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:49:18.949481 27612 raft_consensus.cc:504] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:18.950770 27612 leader_election.cc:288] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4ee92563e1402997d02ba8b1fd8676 (127.15.147.129:33443), c38adac127f84c8bbdb8a91ea391525e (127.15.147.131:35525)
I20260806 00:49:18.955473 27205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.951082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:33443 (local address 127.15.147.130:38687)
0806 00:49:18.951308 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:18.951326 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:18.951439 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:18.951729 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:18.951735 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:18.951758 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:18.951911 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.951920 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.952806 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.952812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.954073 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.954089 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.955029 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.955032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.955152 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.955179 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 00:49:18.955253 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 00:49:18.955330 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260806 00:49:18.955502 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.951201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:38687 (local address 127.15.147.129:33443)
0806 00:49:18.951326 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:18.951331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:18.951488 (+ 157us) server_negotiation.cc:407] Connection header received
0806 00:49:18.951558 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:18.951561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:18.951607 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:18.951696 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:18.952048 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.952684 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.954241 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.954894 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.955017 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.955073 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:18.955283 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 00:49:18.955370 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:49:18.956182 27098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "405c473e8aa147719d81a3ff19937940" candidate_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4ee92563e1402997d02ba8b1fd8676" is_pre_election: true
I20260806 00:49:18.956322 27593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.951305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:51247 (local address 127.15.147.131:35525)
0806 00:49:18.951449 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:18.951454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:18.951836 (+ 382us) server_negotiation.cc:407] Connection header received
0806 00:49:18.951930 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:18.951933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:18.951976 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:18.952044 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:18.952324 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.953106 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.954995 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.955605 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.955735 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.955787 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:18.956114 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 00:49:18.956196 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 00:49:18.956338 27615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.951082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:35525 (local address 127.15.147.130:51247)
0806 00:49:18.951724 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:18.951737 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:18.951838 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:18.952059 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:18.952061 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:18.952080 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:18.952192 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.952198 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.953233 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.953236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.954849 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:18.954859 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.955873 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:18.955879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:18.955981 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:18.956000 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 00:49:18.956092 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 00:49:18.956162 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"mutex_wait_us":272,"thread_start_us":115,"threads_started":1}
I20260806 00:49:18.956557 27098 raft_consensus.cc:2454] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6f3901094e34f41a3d4bdc5436cc8b5 in term 0.
I20260806 00:49:18.956970 27494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "405c473e8aa147719d81a3ff19937940" candidate_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c38adac127f84c8bbdb8a91ea391525e" is_pre_election: true
I20260806 00:49:18.957130 27197 leader_election.cc:302] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4ee92563e1402997d02ba8b1fd8676, f6f3901094e34f41a3d4bdc5436cc8b5; no voters:
I20260806 00:49:18.957274 27494 raft_consensus.cc:2454] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6f3901094e34f41a3d4bdc5436cc8b5 in term 0.
I20260806 00:49:18.957448 27612 raft_consensus.cc:2781] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:49:18.957546 27612 raft_consensus.cc:482] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:49:18.957588 27612 raft_consensus.cc:3037] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:18.958920 27612 raft_consensus.cc:504] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:18.959311 27612 leader_election.cc:288] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [CANDIDATE]: Term 1 election: Requested vote from peers 6a4ee92563e1402997d02ba8b1fd8676 (127.15.147.129:33443), c38adac127f84c8bbdb8a91ea391525e (127.15.147.131:35525)
I20260806 00:49:18.959717 27494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "405c473e8aa147719d81a3ff19937940" candidate_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c38adac127f84c8bbdb8a91ea391525e"
I20260806 00:49:18.959764 27098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "405c473e8aa147719d81a3ff19937940" candidate_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4ee92563e1402997d02ba8b1fd8676"
I20260806 00:49:18.959825 27494 raft_consensus.cc:3037] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:18.959967 27098 raft_consensus.cc:3037] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:18.961205 27098 raft_consensus.cc:2454] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6f3901094e34f41a3d4bdc5436cc8b5 in term 1.
I20260806 00:49:18.961409 27494 raft_consensus.cc:2454] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6f3901094e34f41a3d4bdc5436cc8b5 in term 1.
I20260806 00:49:18.961582 27197 leader_election.cc:302] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4ee92563e1402997d02ba8b1fd8676, f6f3901094e34f41a3d4bdc5436cc8b5; no voters:
I20260806 00:49:18.961787 27612 raft_consensus.cc:2781] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:18.962038 27612 raft_consensus.cc:686] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 LEADER]: Becoming Leader. State: Replica: f6f3901094e34f41a3d4bdc5436cc8b5, State: Running, Role: LEADER
I20260806 00:49:18.962409 27612 consensus_queue.cc:237] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
W20260806 00:49:18.962512 27359 tablet.cc:2369] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:49:18.966068 26870 catalog_manager.cc:5495] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6f3901094e34f41a3d4bdc5436cc8b5 (127.15.147.130). New cstate: current_term: 1 leader_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } health_report { overall_health: UNKNOWN } } }
W20260806 00:49:18.978358 27160 tablet.cc:2369] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:49:19.001665 27588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:18.998300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55460 (local address 127.15.147.130:39021)
0806 00:49:18.998454 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:18.998458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:18.998472 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:49:18.998552 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:18.998554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:18.998609 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:18.998700 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:49:18.998918 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:18.999410 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.000236 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.000534 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.000627 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.000853 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:49:19.001275 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:49:19.001376 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:19.001434 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 00:49:19.001485 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:49:19.007525 27494 raft_consensus.cc:1261] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e [term 1 FOLLOWER]: Refusing update from remote peer f6f3901094e34f41a3d4bdc5436cc8b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:19.007525 27098 raft_consensus.cc:1261] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 FOLLOWER]: Refusing update from remote peer f6f3901094e34f41a3d4bdc5436cc8b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:19.008224 27616 consensus_queue.cc:1035] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:19.008471 27612 consensus_queue.cc:1035] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:19.026495 26866 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:49:19.026679 26866 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:49:19.029287 26866 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35: Sending DeleteTablet for 3 replicas of tablet 405c473e8aa147719d81a3ff19937940
I20260806 00:49:19.030139 27258 tablet_service.cc:1514] Processing DeleteTablet for tablet 405c473e8aa147719d81a3ff19937940 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:19 UTC) from {username='oryx', principal='oryx/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:55456
I20260806 00:49:19.030145 27462 tablet_service.cc:1514] Processing DeleteTablet for tablet 405c473e8aa147719d81a3ff19937940 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:19 UTC) from {username='oryx', principal='oryx/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:39850
I20260806 00:49:19.030656 27066 tablet_service.cc:1514] Processing DeleteTablet for tablet 405c473e8aa147719d81a3ff19937940 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:19 UTC) from {username='oryx', principal='oryx/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:57214
I20260806 00:49:19.030920 26866 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 00:49:19.031230 27632 tablet_replica.cc:330] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e: stopping tablet replica
W20260806 00:49:19.031379 26866 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:49:19.031497 27636 tablet_replica.cc:330] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676: stopping tablet replica
I20260806 00:49:19.031574 27632 raft_consensus.cc:2229] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:19.031939 27632 raft_consensus.cc:2258] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:19.031912 27636 raft_consensus.cc:2229] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:19.032646 27636 raft_consensus.cc:2258] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:19.034667 27636 ts_tablet_manager.cc:1905] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:19.034781 27634 tablet_replica.cc:330] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5: stopping tablet replica
I20260806 00:49:19.035082 27634 raft_consensus.cc:2229] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:49:19.035495 27634 raft_consensus.cc:2258] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:19.037256 27634 ts_tablet_manager.cc:1905] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:19.038012 27636 ts_tablet_manager.cc:1918] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:49:19.038188 27636 log.cc:1196] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/wal/wals/405c473e8aa147719d81a3ff19937940
I20260806 00:49:19.038662 27636 ts_tablet_manager.cc:1939] T 405c473e8aa147719d81a3ff19937940 P 6a4ee92563e1402997d02ba8b1fd8676: Deleting consensus metadata
I20260806 00:49:19.039346 27066 tablet_service.cc:1467] Processing CreateTablet for tablet 291b48b2fbd345568779152cc5e482cc (DEFAULT_TABLE table=test-table [id=c172b092c92d453daeb01021a899ad02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:19.039498 27462 tablet_service.cc:1467] Processing CreateTablet for tablet 291b48b2fbd345568779152cc5e482cc (DEFAULT_TABLE table=test-table [id=c172b092c92d453daeb01021a899ad02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:19.039494 27258 tablet_service.cc:1467] Processing CreateTablet for tablet 291b48b2fbd345568779152cc5e482cc (DEFAULT_TABLE table=test-table [id=c172b092c92d453daeb01021a899ad02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:19.039609 27066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 291b48b2fbd345568779152cc5e482cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:19.039744 27462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 291b48b2fbd345568779152cc5e482cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:19.041735 27596 tablet_bootstrap.cc:492] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676: Bootstrap starting.
I20260806 00:49:19.042837 27596 tablet_bootstrap.cc:654] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:19.043582 27258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 291b48b2fbd345568779152cc5e482cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:19.043983 27634 ts_tablet_manager.cc:1918] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:49:19.044085 27634 log.cc:1196] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/405c473e8aa147719d81a3ff19937940
I20260806 00:49:19.044327 26852 catalog_manager.cc:4841] TS 6a4ee92563e1402997d02ba8b1fd8676 (127.15.147.129:33443): tablet 405c473e8aa147719d81a3ff19937940 (table test-table [id=4bc62d710e6c4e8fbd96190e85473f20]) successfully deleted
I20260806 00:49:19.044427 27634 ts_tablet_manager.cc:1939] T 405c473e8aa147719d81a3ff19937940 P f6f3901094e34f41a3d4bdc5436cc8b5: Deleting consensus metadata
I20260806 00:49:19.045435 26852 catalog_manager.cc:4841] TS f6f3901094e34f41a3d4bdc5436cc8b5 (127.15.147.130:39021): tablet 405c473e8aa147719d81a3ff19937940 (table test-table [id=4bc62d710e6c4e8fbd96190e85473f20]) successfully deleted
I20260806 00:49:19.045914 27602 tablet_bootstrap.cc:492] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5: Bootstrap starting.
I20260806 00:49:19.042465 27610 tablet_bootstrap.cc:492] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e: Bootstrap starting.
I20260806 00:49:19.046901 27602 tablet_bootstrap.cc:654] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:19.046921 27610 tablet_bootstrap.cc:654] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:19.041576 27632 ts_tablet_manager.cc:1905] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:19.048151 27602 tablet_bootstrap.cc:492] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5: No bootstrap required, opened a new log
I20260806 00:49:19.048233 27602 ts_tablet_manager.cc:1397] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 00:49:19.048697 27602 raft_consensus.cc:348] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.048820 27602 raft_consensus.cc:374] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:19.048849 27602 raft_consensus.cc:729] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6f3901094e34f41a3d4bdc5436cc8b5, State: Initialized, Role: FOLLOWER
I20260806 00:49:19.048975 27602 consensus_queue.cc:260] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.049247 27602 ts_tablet_manager.cc:1428] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 00:49:19.049682 27632 ts_tablet_manager.cc:1918] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:49:19.049813 27632 log.cc:1196] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/405c473e8aa147719d81a3ff19937940
I20260806 00:49:19.050114 27632 ts_tablet_manager.cc:1939] T 405c473e8aa147719d81a3ff19937940 P c38adac127f84c8bbdb8a91ea391525e: Deleting consensus metadata
I20260806 00:49:19.051120 27596 tablet_bootstrap.cc:492] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676: No bootstrap required, opened a new log
I20260806 00:49:19.051220 27596 ts_tablet_manager.cc:1397] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260806 00:49:19.051836 26850 catalog_manager.cc:4841] TS c38adac127f84c8bbdb8a91ea391525e (127.15.147.131:35525): tablet 405c473e8aa147719d81a3ff19937940 (table test-table [id=4bc62d710e6c4e8fbd96190e85473f20]) successfully deleted
I20260806 00:49:19.052090 27596 raft_consensus.cc:348] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.052202 27596 raft_consensus.cc:374] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:19.052241 27596 raft_consensus.cc:729] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4ee92563e1402997d02ba8b1fd8676, State: Initialized, Role: FOLLOWER
I20260806 00:49:19.052385 27596 consensus_queue.cc:260] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.052634 27596 ts_tablet_manager.cc:1428] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 00:49:19.053858 27610 tablet_bootstrap.cc:492] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e: No bootstrap required, opened a new log
I20260806 00:49:19.053951 27610 ts_tablet_manager.cc:1397] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20260806 00:49:19.054374 27610 raft_consensus.cc:348] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.054462 27610 raft_consensus.cc:374] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:19.054483 27610 raft_consensus.cc:729] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c38adac127f84c8bbdb8a91ea391525e, State: Initialized, Role: FOLLOWER
I20260806 00:49:19.054617 27610 consensus_queue.cc:260] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.054888 27610 ts_tablet_manager.cc:1428] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 00:49:19.069283 27612 raft_consensus.cc:482] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:49:19.069422 27612 raft_consensus.cc:504] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.069818 27612 leader_election.cc:288] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4ee92563e1402997d02ba8b1fd8676 (127.15.147.129:33443), c38adac127f84c8bbdb8a91ea391525e (127.15.147.131:35525)
I20260806 00:49:19.070161 27494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "291b48b2fbd345568779152cc5e482cc" candidate_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c38adac127f84c8bbdb8a91ea391525e" is_pre_election: true
I20260806 00:49:19.070173 27098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "291b48b2fbd345568779152cc5e482cc" candidate_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4ee92563e1402997d02ba8b1fd8676" is_pre_election: true
I20260806 00:49:19.070300 27098 raft_consensus.cc:2454] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6f3901094e34f41a3d4bdc5436cc8b5 in term 0.
I20260806 00:49:19.070304 27494 raft_consensus.cc:2454] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6f3901094e34f41a3d4bdc5436cc8b5 in term 0.
I20260806 00:49:19.070638 27197 leader_election.cc:302] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4ee92563e1402997d02ba8b1fd8676, f6f3901094e34f41a3d4bdc5436cc8b5; no voters:
I20260806 00:49:19.070899 27612 raft_consensus.cc:2781] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:49:19.070955 27612 raft_consensus.cc:482] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:49:19.070991 27612 raft_consensus.cc:3037] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:19.072208 27612 raft_consensus.cc:504] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.072573 27612 leader_election.cc:288] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [CANDIDATE]: Term 1 election: Requested vote from peers 6a4ee92563e1402997d02ba8b1fd8676 (127.15.147.129:33443), c38adac127f84c8bbdb8a91ea391525e (127.15.147.131:35525)
I20260806 00:49:19.072925 27098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "291b48b2fbd345568779152cc5e482cc" candidate_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4ee92563e1402997d02ba8b1fd8676"
I20260806 00:49:19.072927 27494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "291b48b2fbd345568779152cc5e482cc" candidate_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c38adac127f84c8bbdb8a91ea391525e"
I20260806 00:49:19.073045 27494 raft_consensus.cc:3037] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:19.073045 27098 raft_consensus.cc:3037] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:19.074059 27098 raft_consensus.cc:2454] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6f3901094e34f41a3d4bdc5436cc8b5 in term 1.
I20260806 00:49:19.074112 27494 raft_consensus.cc:2454] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6f3901094e34f41a3d4bdc5436cc8b5 in term 1.
I20260806 00:49:19.074404 27197 leader_election.cc:302] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4ee92563e1402997d02ba8b1fd8676, f6f3901094e34f41a3d4bdc5436cc8b5; no voters:
I20260806 00:49:19.074604 27612 raft_consensus.cc:2781] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:19.074697 27612 raft_consensus.cc:686] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 LEADER]: Becoming Leader. State: Replica: f6f3901094e34f41a3d4bdc5436cc8b5, State: Running, Role: LEADER
I20260806 00:49:19.074842 27612 consensus_queue.cc:237] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.076490 26866 catalog_manager.cc:5495] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6f3901094e34f41a3d4bdc5436cc8b5 (127.15.147.130). New cstate: current_term: 1 leader_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } health_report { overall_health: UNKNOWN } } }
Aug 06 00:49:19 dist-test-slave-45dx krb5kdc[26811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977357, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.190@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM
I20260806 00:49:19.122721 27652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.114134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:41237 (local address 127.0.0.1:44820)
0806 00:49:19.114425 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:19.114436 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:19.114564 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:19.114826 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:19.114828 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:19.115273 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:19.115383 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.115391 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.116146 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.116149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.116758 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.116767 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.117196 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.117199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.117273 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.117900 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:19.117910 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:19.119508 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:19.121184 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:19.121195 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:19.121201 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:19.121425 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:19.121683 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:19.121688 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:19.121690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:19.121780 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:19.122135 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:19.122143 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:19.122248 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 00:49:19.122357 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 00:49:19.122509 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":113,"threads_started":1}
I20260806 00:49:19.122720 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.114181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44820 (local address 127.15.147.190:41237)
0806 00:49:19.114325 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:19.114331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:19.114509 (+ 178us) server_negotiation.cc:407] Connection header received
0806 00:49:19.114686 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:19.114689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:19.114730 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:19.114808 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:19.115495 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.116033 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.116870 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.117094 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.117193 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.119642 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:19.119645 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:19.119647 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:19.119666 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:19.121082 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:19.121520 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:19.121525 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:19.121527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:19.121599 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:19.121862 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:19.121866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:19.121867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:19.122035 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:19.122135 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:19.122409 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 00:49:19.122562 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":44}
I20260806 00:49:19.126937 26866 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:44820:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 00:49:19.127633 26866 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:49:19.132776 27258 tablet_service.cc:1467] Processing CreateTablet for tablet 7cedfe4c744441ef9abe6e8d324e6bd4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f0058c05bb4149418e1c564c78ad65ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:49:19.132781 27462 tablet_service.cc:1467] Processing CreateTablet for tablet 7cedfe4c744441ef9abe6e8d324e6bd4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f0058c05bb4149418e1c564c78ad65ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:49:19.132892 27066 tablet_service.cc:1467] Processing CreateTablet for tablet 7cedfe4c744441ef9abe6e8d324e6bd4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f0058c05bb4149418e1c564c78ad65ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 00:49:19.133194 27258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cedfe4c744441ef9abe6e8d324e6bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:19.133194 27462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cedfe4c744441ef9abe6e8d324e6bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:19.133306 27066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cedfe4c744441ef9abe6e8d324e6bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:19.136125 27602 tablet_bootstrap.cc:492] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5: Bootstrap starting.
I20260806 00:49:19.136412 27610 tablet_bootstrap.cc:492] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e: Bootstrap starting.
I20260806 00:49:19.137220 27602 tablet_bootstrap.cc:654] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:19.137315 27610 tablet_bootstrap.cc:654] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:19.138681 27602 tablet_bootstrap.cc:492] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5: No bootstrap required, opened a new log
I20260806 00:49:19.138780 27602 ts_tablet_manager.cc:1397] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:49:19.139590 27602 raft_consensus.cc:348] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.139715 27602 raft_consensus.cc:374] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:19.139755 27602 raft_consensus.cc:729] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6f3901094e34f41a3d4bdc5436cc8b5, State: Initialized, Role: FOLLOWER
I20260806 00:49:19.139882 27602 consensus_queue.cc:260] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.140589 27596 tablet_bootstrap.cc:492] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676: Bootstrap starting.
I20260806 00:49:19.140877 27612 tablet_replica.cc:439] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } } }
I20260806 00:49:19.141027 27612 tablet_replica.cc:442] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:49:19.141098 27610 tablet_bootstrap.cc:492] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e: No bootstrap required, opened a new log
I20260806 00:49:19.141177 27610 ts_tablet_manager.cc:1397] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 00:49:19.141599 27596 tablet_bootstrap.cc:654] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:19.141613 27610 raft_consensus.cc:348] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.141716 27610 raft_consensus.cc:374] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:19.141752 27610 raft_consensus.cc:729] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c38adac127f84c8bbdb8a91ea391525e, State: Initialized, Role: FOLLOWER
I20260806 00:49:19.141934 27610 consensus_queue.cc:260] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.142252 27610 ts_tablet_manager.cc:1428] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:49:19.142506 27614 tablet_replica.cc:439] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } } }
I20260806 00:49:19.142666 27614 tablet_replica.cc:442] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:49:19.142975 27602 ts_tablet_manager.cc:1428] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 00:49:19.144152 27596 tablet_bootstrap.cc:492] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676: No bootstrap required, opened a new log
I20260806 00:49:19.144243 27596 ts_tablet_manager.cc:1397] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 00:49:19.144866 27596 raft_consensus.cc:348] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.145002 27596 raft_consensus.cc:374] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:19.145040 27596 raft_consensus.cc:729] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4ee92563e1402997d02ba8b1fd8676, State: Initialized, Role: FOLLOWER
I20260806 00:49:19.145177 27596 consensus_queue.cc:260] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.145625 27611 tablet_replica.cc:439] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } } }
I20260806 00:49:19.145789 27611 tablet_replica.cc:442] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:49:19.146418 27596 ts_tablet_manager.cc:1428] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 00:49:19.162674 27611 raft_consensus.cc:482] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:49:19.162827 27611 raft_consensus.cc:504] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.163903 27611 leader_election.cc:288] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6f3901094e34f41a3d4bdc5436cc8b5 (127.15.147.130:39021), c38adac127f84c8bbdb8a91ea391525e (127.15.147.131:35525)
I20260806 00:49:19.168768 27588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.164374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:37039 (local address 127.15.147.130:39021)
0806 00:49:19.164548 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:19.164555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:19.164671 (+ 116us) server_negotiation.cc:407] Connection header received
0806 00:49:19.164907 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:19.164912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:19.164974 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:19.165079 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:19.165389 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.166246 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.167621 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.168107 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.168221 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.168274 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:19.168551 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 00:49:19.168636 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 00:49:19.168828 27658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.164125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:39021 (local address 127.15.147.129:37039)
0806 00:49:19.164582 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:19.164597 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:19.164757 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:19.165104 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:19.165107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:19.165123 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:19.165271 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.165277 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.166477 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.166479 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.167498 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.167508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.168233 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.168237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.168387 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.168425 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 00:49:19.168518 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 00:49:19.168609 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":52,"thread_start_us":171,"threads_started":1}
I20260806 00:49:19.169198 27593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.164510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:42905 (local address 127.15.147.131:35525)
0806 00:49:19.164647 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:19.164651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:19.164806 (+ 155us) server_negotiation.cc:407] Connection header received
0806 00:49:19.165008 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:19.165013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:19.165064 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:19.165138 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:19.165533 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.166311 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.167971 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.168524 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.168633 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.168674 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:19.168999 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 00:49:19.169093 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 00:49:19.169391 27283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cedfe4c744441ef9abe6e8d324e6bd4" candidate_uuid: "6a4ee92563e1402997d02ba8b1fd8676" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" is_pre_election: true
I20260806 00:49:19.169674 27283 raft_consensus.cc:2454] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4ee92563e1402997d02ba8b1fd8676 in term 0.
I20260806 00:49:19.170071 27657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.164157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:35525 (local address 127.15.147.129:42905)
0806 00:49:19.164716 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:19.164729 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:19.164849 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:19.165251 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:19.165254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:19.165272 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:19.165419 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.165425 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.166610 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.166614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.167840 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.167852 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.168732 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.168736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.168844 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.168869 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 00:49:19.169417 (+ 548us) client_negotiation.cc:240] Negotiation successful
0806 00:49:19.169492 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"mutex_wait_us":43,"thread_start_us":160,"threads_started":1}
I20260806 00:49:19.170115 27003 leader_election.cc:302] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4ee92563e1402997d02ba8b1fd8676, f6f3901094e34f41a3d4bdc5436cc8b5; no voters:
I20260806 00:49:19.170416 27611 raft_consensus.cc:2781] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:49:19.170485 27611 raft_consensus.cc:482] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:49:19.170529 27611 raft_consensus.cc:3037] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:19.170578 27494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cedfe4c744441ef9abe6e8d324e6bd4" candidate_uuid: "6a4ee92563e1402997d02ba8b1fd8676" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c38adac127f84c8bbdb8a91ea391525e" is_pre_election: true
I20260806 00:49:19.170730 27494 raft_consensus.cc:2454] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a4ee92563e1402997d02ba8b1fd8676 in term 0.
I20260806 00:49:19.171890 27611 raft_consensus.cc:504] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.172221 27611 leader_election.cc:288] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [CANDIDATE]: Term 1 election: Requested vote from peers f6f3901094e34f41a3d4bdc5436cc8b5 (127.15.147.130:39021), c38adac127f84c8bbdb8a91ea391525e (127.15.147.131:35525)
I20260806 00:49:19.172616 27494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cedfe4c744441ef9abe6e8d324e6bd4" candidate_uuid: "6a4ee92563e1402997d02ba8b1fd8676" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c38adac127f84c8bbdb8a91ea391525e"
I20260806 00:49:19.172665 27283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cedfe4c744441ef9abe6e8d324e6bd4" candidate_uuid: "6a4ee92563e1402997d02ba8b1fd8676" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5"
I20260806 00:49:19.172752 27494 raft_consensus.cc:3037] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:19.172791 27283 raft_consensus.cc:3037] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:19.173620 27494 raft_consensus.cc:2454] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4ee92563e1402997d02ba8b1fd8676 in term 1.
I20260806 00:49:19.173911 26999 leader_election.cc:302] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4ee92563e1402997d02ba8b1fd8676, c38adac127f84c8bbdb8a91ea391525e; no voters:
I20260806 00:49:19.174017 27283 raft_consensus.cc:2454] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a4ee92563e1402997d02ba8b1fd8676 in term 1.
I20260806 00:49:19.174094 27611 raft_consensus.cc:2781] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:19.174330 27611 raft_consensus.cc:686] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 LEADER]: Becoming Leader. State: Replica: 6a4ee92563e1402997d02ba8b1fd8676, State: Running, Role: LEADER
I20260806 00:49:19.174592 27611 consensus_queue.cc:237] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.176584 27660 tablet_replica.cc:439] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676: TxnStatusTablet state changed. Reason: New leader 6a4ee92563e1402997d02ba8b1fd8676. Latest consensus state: current_term: 1 leader_uuid: "6a4ee92563e1402997d02ba8b1fd8676" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } } }
I20260806 00:49:19.176689 27660 tablet_replica.cc:442] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:49:19.176759 26867 catalog_manager.cc:5495] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a4ee92563e1402997d02ba8b1fd8676 (127.15.147.129). New cstate: current_term: 1 leader_uuid: "6a4ee92563e1402997d02ba8b1fd8676" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } health_report { overall_health: UNKNOWN } } }
I20260806 00:49:19.176987 27663 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 00:49:19.177266 27663 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 00:49:19.197729 27652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.192831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:33443 (local address 127.0.0.1:57224)
0806 00:49:19.193005 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:19.193023 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:19.193136 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:19.193534 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:19.193537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:19.193545 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:49:19.193671 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.193677 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.194638 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.194644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.195624 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.195636 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.196404 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.196409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.196505 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.196560 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:49:19.197376 (+ 816us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:49:19.197406 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 00:49:19.197474 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 00:49:19.197568 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 00:49:19.198244 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.193081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57224 (local address 127.15.147.129:33443)
0806 00:49:19.193232 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:19.193236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:19.193251 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:49:19.193335 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:19.193338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:19.193400 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:19.193499 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:49:19.193777 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.194532 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.195795 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.196148 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.196262 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.196715 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:49:19.197266 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:49:19.197382 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:19.198059 (+ 677us) server_negotiation.cc:299] Negotiation successful
0806 00:49:19.198122 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:49:19.201285 27494 raft_consensus.cc:1261] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e [term 1 FOLLOWER]: Refusing update from remote peer 6a4ee92563e1402997d02ba8b1fd8676: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:19.201550 27283 raft_consensus.cc:1261] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 FOLLOWER]: Refusing update from remote peer 6a4ee92563e1402997d02ba8b1fd8676: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:19.201779 27660 consensus_queue.cc:1035] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:19.202075 27611 consensus_queue.cc:1035] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:19.203665 27612 tablet_replica.cc:439] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5: TxnStatusTablet state changed. Reason: New leader 6a4ee92563e1402997d02ba8b1fd8676. Latest consensus state: current_term: 1 leader_uuid: "6a4ee92563e1402997d02ba8b1fd8676" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } } }
I20260806 00:49:19.203797 27612 tablet_replica.cc:442] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:49:19.205024 27614 tablet_replica.cc:439] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e: TxnStatusTablet state changed. Reason: New leader 6a4ee92563e1402997d02ba8b1fd8676. Latest consensus state: current_term: 1 leader_uuid: "6a4ee92563e1402997d02ba8b1fd8676" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } } }
I20260806 00:49:19.205150 27614 tablet_replica.cc:442] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:49:19.206441 27611 tablet_replica.cc:439] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6a4ee92563e1402997d02ba8b1fd8676" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } } }
I20260806 00:49:19.206555 27611 tablet_replica.cc:442] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:49:19.207059 27622 mvcc.cc:205] Tried to move back new op lower bound from 7315363263283105792 to 7315363263181340672. Current Snapshot: MvccSnapshot[applied={T|T < 7315363263283105792 or (T in {7315363263283105792})}]
I20260806 00:49:19.207353 27612 tablet_replica.cc:439] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6a4ee92563e1402997d02ba8b1fd8676" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } } }
I20260806 00:49:19.207453 27612 tablet_replica.cc:442] T 7cedfe4c744441ef9abe6e8d324e6bd4 P f6f3901094e34f41a3d4bdc5436cc8b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:49:19.208180 27660 tablet_replica.cc:439] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6a4ee92563e1402997d02ba8b1fd8676" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } } }
I20260806 00:49:19.208284 27660 tablet_replica.cc:442] T 7cedfe4c744441ef9abe6e8d324e6bd4 P 6a4ee92563e1402997d02ba8b1fd8676: This TxnStatusTablet replica's current role is: LEADER
I20260806 00:49:19.208974 27623 mvcc.cc:205] Tried to move back new op lower bound from 7315363263283105792 to 7315363263181340672. Current Snapshot: MvccSnapshot[applied={T|T < 7315363263283105792}]
I20260806 00:49:19.209733 27614 tablet_replica.cc:439] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6a4ee92563e1402997d02ba8b1fd8676" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } } }
I20260806 00:49:19.209911 27614 tablet_replica.cc:442] T 7cedfe4c744441ef9abe6e8d324e6bd4 P c38adac127f84c8bbdb8a91ea391525e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 00:49:19.222767 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.219175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:33443 (local address 127.15.147.130:41381)
0806 00:49:19.219522 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:19.219535 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:19.219658 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:19.219926 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:19.219929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:19.219950 (+ 21us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:49:19.220069 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.220073 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.220852 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.220857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.221477 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.221485 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.221983 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.221986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.222099 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.222143 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:49:19.222455 (+ 312us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:49:19.222464 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 00:49:19.222528 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 00:49:19.222626 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":107,"threads_started":1}
I20260806 00:49:19.222751 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.219274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:41381 (local address 127.15.147.129:33443)
0806 00:49:19.219411 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:19.219416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:19.219609 (+ 193us) server_negotiation.cc:407] Connection header received
0806 00:49:19.219762 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:19.219765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:19.219819 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:19.219896 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:49:19.220172 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.220746 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.221616 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.221866 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.221975 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.222290 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:49:19.222373 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:49:19.222451 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:19.222546 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 00:49:19.222601 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:49:19.230921 27098 raft_consensus.cc:1261] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 FOLLOWER]: Refusing update from remote peer f6f3901094e34f41a3d4bdc5436cc8b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:19.230944 27494 raft_consensus.cc:1261] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e [term 1 FOLLOWER]: Refusing update from remote peer f6f3901094e34f41a3d4bdc5436cc8b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:19.231345 27616 consensus_queue.cc:1035] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:19.231813 27616 consensus_queue.cc:1035] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:19.236657 27624 mvcc.cc:205] Tried to move back new op lower bound from 7315363263406186496 to 7315363262772060160. Current Snapshot: MvccSnapshot[applied={T|T < 7315363263406186496}]
I20260806 00:49:19.257965 27588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.254107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:50947 (local address 127.15.147.130:39021)
0806 00:49:19.254233 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:19.254239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:19.254474 (+ 235us) server_negotiation.cc:407] Connection header received
0806 00:49:19.254626 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:19.254629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:19.254674 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:19.254755 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:49:19.255039 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.255712 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.256852 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.257092 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.257196 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.257503 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:49:19.257582 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:49:19.257655 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:19.257759 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 00:49:19.257829 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 00:49:19.257980 27678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.254055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:39021 (local address 127.15.147.129:50947)
0806 00:49:19.254392 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:19.254405 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:19.254522 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:19.254765 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:19.254768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:19.254776 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:49:19.254936 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.254942 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.255852 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.255855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.256725 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.256736 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.257209 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.257213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.257320 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.257366 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:49:19.257674 (+ 308us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:49:19.257682 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 00:49:19.257740 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 00:49:19.257830 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":116,"threads_started":1}
I20260806 00:49:19.285672 26870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:49:19.285872 26870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:49:19.288018 26870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35: Sending DeleteTablet for 3 replicas of tablet 291b48b2fbd345568779152cc5e482cc
I20260806 00:49:19.288635 27462 tablet_service.cc:1514] Processing DeleteTablet for tablet 291b48b2fbd345568779152cc5e482cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:19 UTC) from {username='oryx', principal='oryx/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:39850
I20260806 00:49:19.288635 27066 tablet_service.cc:1514] Processing DeleteTablet for tablet 291b48b2fbd345568779152cc5e482cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:19 UTC) from {username='oryx', principal='oryx/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:57214
I20260806 00:49:19.288820 27632 tablet_replica.cc:330] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e: stopping tablet replica
I20260806 00:49:19.288839 27636 tablet_replica.cc:330] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676: stopping tablet replica
I20260806 00:49:19.288925 27632 raft_consensus.cc:2229] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:19.288975 27636 raft_consensus.cc:2229] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:19.289036 27632 raft_consensus.cc:2258] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:19.289088 27636 raft_consensus.cc:2258] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:19.289904 27632 ts_tablet_manager.cc:1905] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:19.290082 27636 ts_tablet_manager.cc:1905] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:19.292999 27632 ts_tablet_manager.cc:1918] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:49:19.293084 27632 log.cc:1196] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/291b48b2fbd345568779152cc5e482cc
I20260806 00:49:19.293375 27632 ts_tablet_manager.cc:1939] T 291b48b2fbd345568779152cc5e482cc P c38adac127f84c8bbdb8a91ea391525e: Deleting consensus metadata
I20260806 00:49:19.293678 27636 ts_tablet_manager.cc:1918] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:49:19.293759 27636 log.cc:1196] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/wal/wals/291b48b2fbd345568779152cc5e482cc
I20260806 00:49:19.294049 27636 ts_tablet_manager.cc:1939] T 291b48b2fbd345568779152cc5e482cc P 6a4ee92563e1402997d02ba8b1fd8676: Deleting consensus metadata
I20260806 00:49:19.294099 26850 catalog_manager.cc:4841] TS c38adac127f84c8bbdb8a91ea391525e (127.15.147.131:35525): tablet 291b48b2fbd345568779152cc5e482cc (table test-table [id=c172b092c92d453daeb01021a899ad02]) successfully deleted
I20260806 00:49:19.294492 27258 tablet_service.cc:1514] Processing DeleteTablet for tablet 291b48b2fbd345568779152cc5e482cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:19 UTC) from {username='oryx', principal='oryx/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:55456
I20260806 00:49:19.294674 27634 tablet_replica.cc:330] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5: stopping tablet replica
I20260806 00:49:19.294793 27634 raft_consensus.cc:2229] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:49:19.294765 26852 catalog_manager.cc:4841] TS 6a4ee92563e1402997d02ba8b1fd8676 (127.15.147.129:33443): tablet 291b48b2fbd345568779152cc5e482cc (table test-table [id=c172b092c92d453daeb01021a899ad02]) successfully deleted
I20260806 00:49:19.294984 27634 raft_consensus.cc:2258] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:19.295805 27634 ts_tablet_manager.cc:1905] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:19.298700 27634 ts_tablet_manager.cc:1918] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:49:19.298799 27634 log.cc:1196] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/291b48b2fbd345568779152cc5e482cc
I20260806 00:49:19.299091 27634 ts_tablet_manager.cc:1939] T 291b48b2fbd345568779152cc5e482cc P f6f3901094e34f41a3d4bdc5436cc8b5: Deleting consensus metadata
I20260806 00:49:19.299943 26852 catalog_manager.cc:4841] TS f6f3901094e34f41a3d4bdc5436cc8b5 (127.15.147.130:39021): tablet 291b48b2fbd345568779152cc5e482cc (table test-table [id=c172b092c92d453daeb01021a899ad02]) successfully deleted
Aug 06 00:49:19 dist-test-slave-45dx krb5kdc[26811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 00:49:19 dist-test-slave-45dx krb5kdc[26811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:19 dist-test-slave-45dx krb5kdc[26811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 00:49:19.321134 26940 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:19.314336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44836 (local address 127.15.147.190:41237)
0806 00:49:19.314471 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:19.314475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:19.314745 (+ 270us) server_negotiation.cc:407] Connection header received
0806 00:49:19.314918 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:19.314922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:19.314979 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:19.315069 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:19.315900 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.316555 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.317430 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.317687 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.317799 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.320994 (+ 3195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44836: BlockingRecv error: recv got EOF from 127.0.0.1:44836 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 00:49:19 dist-test-slave-45dx krb5kdc[26811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM
I20260806 00:49:19.332619 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.323060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44850 (local address 127.15.147.190:41237)
0806 00:49:19.323215 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:19.323220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:19.323316 (+ 96us) server_negotiation.cc:407] Connection header received
0806 00:49:19.323413 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:19.323416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:19.323468 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:19.323556 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:19.324149 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.324949 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.325844 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.326211 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.326347 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.328963 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:19.328968 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:19.328971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:19.329003 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:19.330689 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:19.331130 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:19.331134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:19.331136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:19.331244 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:19.331557 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:19.331561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:19.331567 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:19.331821 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:19.331971 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:19.332236 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 00:49:19.332412 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 00:49:19.335678 26870 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:49:19.336195 26870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:49:19.341326 27462 tablet_service.cc:1467] Processing CreateTablet for tablet 32433ff500dd4ce5884808a70db71f35 (DEFAULT_TABLE table=test-table [id=590c332b94c44dda93f0e7fe5594a9d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:19.341430 27258 tablet_service.cc:1467] Processing CreateTablet for tablet 32433ff500dd4ce5884808a70db71f35 (DEFAULT_TABLE table=test-table [id=590c332b94c44dda93f0e7fe5594a9d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:19.341677 27462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32433ff500dd4ce5884808a70db71f35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:19.341792 27258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32433ff500dd4ce5884808a70db71f35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:19.344455 27602 tablet_bootstrap.cc:492] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5: Bootstrap starting.
I20260806 00:49:19.344455 27610 tablet_bootstrap.cc:492] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e: Bootstrap starting.
I20260806 00:49:19.345331 27610 tablet_bootstrap.cc:654] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:19.345331 27602 tablet_bootstrap.cc:654] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:19.346174 27066 tablet_service.cc:1467] Processing CreateTablet for tablet 32433ff500dd4ce5884808a70db71f35 (DEFAULT_TABLE table=test-table [id=590c332b94c44dda93f0e7fe5594a9d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:19.346469 27066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32433ff500dd4ce5884808a70db71f35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:19.346689 27602 tablet_bootstrap.cc:492] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5: No bootstrap required, opened a new log
I20260806 00:49:19.346774 27602 ts_tablet_manager.cc:1397] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 00:49:19.347199 27602 raft_consensus.cc:348] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.347316 27602 raft_consensus.cc:374] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:19.347344 27602 raft_consensus.cc:729] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6f3901094e34f41a3d4bdc5436cc8b5, State: Initialized, Role: FOLLOWER
I20260806 00:49:19.347470 27602 consensus_queue.cc:260] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.347736 27602 ts_tablet_manager.cc:1428] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 00:49:19.349188 27596 tablet_bootstrap.cc:492] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676: Bootstrap starting.
I20260806 00:49:19.349936 27610 tablet_bootstrap.cc:492] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e: No bootstrap required, opened a new log
I20260806 00:49:19.350034 27610 ts_tablet_manager.cc:1397] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260806 00:49:19.350147 27596 tablet_bootstrap.cc:654] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:19.350551 27610 raft_consensus.cc:348] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.350695 27610 raft_consensus.cc:374] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:19.350734 27610 raft_consensus.cc:729] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c38adac127f84c8bbdb8a91ea391525e, State: Initialized, Role: FOLLOWER
I20260806 00:49:19.350868 27610 consensus_queue.cc:260] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.351156 27610 ts_tablet_manager.cc:1428] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:49:19.352473 27596 tablet_bootstrap.cc:492] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676: No bootstrap required, opened a new log
I20260806 00:49:19.352555 27596 ts_tablet_manager.cc:1397] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 00:49:19.353009 27596 raft_consensus.cc:348] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.353142 27596 raft_consensus.cc:374] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:19.353179 27596 raft_consensus.cc:729] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4ee92563e1402997d02ba8b1fd8676, State: Initialized, Role: FOLLOWER
I20260806 00:49:19.353291 27596 consensus_queue.cc:260] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.353538 27596 ts_tablet_manager.cc:1428] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 00:49:19.470999 27616 raft_consensus.cc:482] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:49:19.471143 27616 raft_consensus.cc:504] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.471609 27616 leader_election.cc:288] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4ee92563e1402997d02ba8b1fd8676 (127.15.147.129:33443), c38adac127f84c8bbdb8a91ea391525e (127.15.147.131:35525)
I20260806 00:49:19.472157 27098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32433ff500dd4ce5884808a70db71f35" candidate_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4ee92563e1402997d02ba8b1fd8676" is_pre_election: true
I20260806 00:49:19.472370 27098 raft_consensus.cc:2454] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6f3901094e34f41a3d4bdc5436cc8b5 in term 0.
I20260806 00:49:19.472599 27493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32433ff500dd4ce5884808a70db71f35" candidate_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c38adac127f84c8bbdb8a91ea391525e" is_pre_election: true
I20260806 00:49:19.472730 27197 leader_election.cc:302] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4ee92563e1402997d02ba8b1fd8676, f6f3901094e34f41a3d4bdc5436cc8b5; no voters:
I20260806 00:49:19.472779 27493 raft_consensus.cc:2454] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6f3901094e34f41a3d4bdc5436cc8b5 in term 0.
I20260806 00:49:19.472900 27616 raft_consensus.cc:2781] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:49:19.472949 27616 raft_consensus.cc:482] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:49:19.472992 27616 raft_consensus.cc:3037] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:19.474259 27616 raft_consensus.cc:504] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.474606 27616 leader_election.cc:288] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [CANDIDATE]: Term 1 election: Requested vote from peers 6a4ee92563e1402997d02ba8b1fd8676 (127.15.147.129:33443), c38adac127f84c8bbdb8a91ea391525e (127.15.147.131:35525)
I20260806 00:49:19.474985 27098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32433ff500dd4ce5884808a70db71f35" candidate_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4ee92563e1402997d02ba8b1fd8676"
I20260806 00:49:19.475004 27493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32433ff500dd4ce5884808a70db71f35" candidate_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c38adac127f84c8bbdb8a91ea391525e"
I20260806 00:49:19.475116 27493 raft_consensus.cc:3037] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:19.475124 27098 raft_consensus.cc:3037] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:19.476074 27098 raft_consensus.cc:2454] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6f3901094e34f41a3d4bdc5436cc8b5 in term 1.
I20260806 00:49:19.476392 27197 leader_election.cc:302] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4ee92563e1402997d02ba8b1fd8676, f6f3901094e34f41a3d4bdc5436cc8b5; no voters:
I20260806 00:49:19.476617 27493 raft_consensus.cc:2454] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6f3901094e34f41a3d4bdc5436cc8b5 in term 1.
I20260806 00:49:19.476810 27616 raft_consensus.cc:2781] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:19.476893 27616 raft_consensus.cc:686] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 LEADER]: Becoming Leader. State: Replica: f6f3901094e34f41a3d4bdc5436cc8b5, State: Running, Role: LEADER
I20260806 00:49:19.477022 27616 consensus_queue.cc:237] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.478816 26866 catalog_manager.cc:5495] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6f3901094e34f41a3d4bdc5436cc8b5 (127.15.147.130). New cstate: current_term: 1 leader_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } health_report { overall_health: UNKNOWN } } }
I20260806 00:49:19.509250 27588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.506067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55466 (local address 127.15.147.130:39021)
0806 00:49:19.506216 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:19.506221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:19.506235 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:49:19.506367 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:19.506371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:19.506420 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:19.506500 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:49:19.506760 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.507277 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.508141 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.508435 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.508546 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.508802 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:49:19.508881 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:49:19.508958 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:19.509054 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 00:49:19.509109 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:49:19.511272 27098 raft_consensus.cc:1261] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 FOLLOWER]: Refusing update from remote peer f6f3901094e34f41a3d4bdc5436cc8b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:19.511278 27493 raft_consensus.cc:1261] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e [term 1 FOLLOWER]: Refusing update from remote peer f6f3901094e34f41a3d4bdc5436cc8b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:19.511735 27616 consensus_queue.cc:1035] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:19.511900 27612 consensus_queue.cc:1035] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:19.519869 26870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:49:19.520020 26870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:49:19.522286 26870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35: Sending DeleteTablet for 3 replicas of tablet 32433ff500dd4ce5884808a70db71f35
I20260806 00:49:19.522832 27066 tablet_service.cc:1514] Processing DeleteTablet for tablet 32433ff500dd4ce5884808a70db71f35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:19 UTC) from {username='oryx', principal='oryx/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:57214
I20260806 00:49:19.522874 27258 tablet_service.cc:1514] Processing DeleteTablet for tablet 32433ff500dd4ce5884808a70db71f35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:19 UTC) from {username='oryx', principal='oryx/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:55456
I20260806 00:49:19.523046 27636 tablet_replica.cc:330] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676: stopping tablet replica
I20260806 00:49:19.523083 27634 tablet_replica.cc:330] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5: stopping tablet replica
I20260806 00:49:19.523075 27462 tablet_service.cc:1514] Processing DeleteTablet for tablet 32433ff500dd4ce5884808a70db71f35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:19 UTC) from {username='oryx', principal='oryx/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:39850
I20260806 00:49:19.523183 27636 raft_consensus.cc:2229] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:19.523226 27634 raft_consensus.cc:2229] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:49:19.523289 27461 tablet_service.cc:1514] Processing DeleteTablet for tablet 32433ff500dd4ce5884808a70db71f35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:19 UTC) from {username='oryx', principal='oryx/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:39850
I20260806 00:49:19.523324 27636 raft_consensus.cc:2258] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:19.523336 27632 tablet_replica.cc:330] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e: stopping tablet replica
I20260806 00:49:19.523412 27634 raft_consensus.cc:2258] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:19.523458 27632 raft_consensus.cc:2229] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:19.523567 27632 raft_consensus.cc:2258] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:19.524058 27636 ts_tablet_manager.cc:1905] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:19.524259 27634 ts_tablet_manager.cc:1905] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:19.524586 26870 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 00:49:19.524863 27632 ts_tablet_manager.cc:1905] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 00:49:19.525138 26870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:49:19.526822 27634 ts_tablet_manager.cc:1918] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:49:19.526813 27636 ts_tablet_manager.cc:1918] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:49:19.526919 27636 log.cc:1196] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/wal/wals/32433ff500dd4ce5884808a70db71f35
I20260806 00:49:19.526921 27634 log.cc:1196] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/32433ff500dd4ce5884808a70db71f35
I20260806 00:49:19.527218 27634 ts_tablet_manager.cc:1939] T 32433ff500dd4ce5884808a70db71f35 P f6f3901094e34f41a3d4bdc5436cc8b5: Deleting consensus metadata
I20260806 00:49:19.527217 27636 ts_tablet_manager.cc:1939] T 32433ff500dd4ce5884808a70db71f35 P 6a4ee92563e1402997d02ba8b1fd8676: Deleting consensus metadata
I20260806 00:49:19.527485 27632 ts_tablet_manager.cc:1918] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:49:19.527568 27632 log.cc:1196] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/32433ff500dd4ce5884808a70db71f35
I20260806 00:49:19.527829 27632 ts_tablet_manager.cc:1939] T 32433ff500dd4ce5884808a70db71f35 P c38adac127f84c8bbdb8a91ea391525e: Deleting consensus metadata
I20260806 00:49:19.527969 26852 catalog_manager.cc:4841] TS 6a4ee92563e1402997d02ba8b1fd8676 (127.15.147.129:33443): tablet 32433ff500dd4ce5884808a70db71f35 (table test-table [id=590c332b94c44dda93f0e7fe5594a9d2]) successfully deleted
I20260806 00:49:19.528534 26850 catalog_manager.cc:4841] TS c38adac127f84c8bbdb8a91ea391525e (127.15.147.131:35525): tablet 32433ff500dd4ce5884808a70db71f35 (table test-table [id=590c332b94c44dda93f0e7fe5594a9d2]) successfully deleted
W20260806 00:49:19.528844 26850 catalog_manager.cc:4808] TS c38adac127f84c8bbdb8a91ea391525e (127.15.147.131:35525): delete failed for tablet 32433ff500dd4ce5884808a70db71f35 because the tablet was not found. No further retry: Not found: Tablet not found: 32433ff500dd4ce5884808a70db71f35
I20260806 00:49:19.529309 26852 catalog_manager.cc:4841] TS f6f3901094e34f41a3d4bdc5436cc8b5 (127.15.147.130:39021): tablet 32433ff500dd4ce5884808a70db71f35 (table test-table [id=590c332b94c44dda93f0e7fe5594a9d2]) successfully deleted
I20260806 00:49:19.531893 27066 tablet_service.cc:1467] Processing CreateTablet for tablet 50529b7a2cb74fb3abbd94ef2f2c8236 (DEFAULT_TABLE table=test-table [id=bf7628c0b91d4fcaa433bb3d8909a7b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:19.531875 27258 tablet_service.cc:1467] Processing CreateTablet for tablet 50529b7a2cb74fb3abbd94ef2f2c8236 (DEFAULT_TABLE table=test-table [id=bf7628c0b91d4fcaa433bb3d8909a7b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:19.532033 27461 tablet_service.cc:1467] Processing CreateTablet for tablet 50529b7a2cb74fb3abbd94ef2f2c8236 (DEFAULT_TABLE table=test-table [id=bf7628c0b91d4fcaa433bb3d8909a7b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:19.532166 27258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50529b7a2cb74fb3abbd94ef2f2c8236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:19.532166 27066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50529b7a2cb74fb3abbd94ef2f2c8236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:19.532356 27461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50529b7a2cb74fb3abbd94ef2f2c8236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:19.534278 27610 tablet_bootstrap.cc:492] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e: Bootstrap starting.
I20260806 00:49:19.534510 27596 tablet_bootstrap.cc:492] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676: Bootstrap starting.
I20260806 00:49:19.535050 27610 tablet_bootstrap.cc:654] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:19.535180 27596 tablet_bootstrap.cc:654] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:19.535804 27602 tablet_bootstrap.cc:492] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5: Bootstrap starting.
I20260806 00:49:19.536165 27610 tablet_bootstrap.cc:492] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e: No bootstrap required, opened a new log
I20260806 00:49:19.536273 27610 ts_tablet_manager.cc:1397] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 00:49:19.536504 27602 tablet_bootstrap.cc:654] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:19.536734 27610 raft_consensus.cc:348] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.536849 27610 raft_consensus.cc:374] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:19.536890 27610 raft_consensus.cc:729] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c38adac127f84c8bbdb8a91ea391525e, State: Initialized, Role: FOLLOWER
I20260806 00:49:19.537045 27610 consensus_queue.cc:260] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.537312 27610 ts_tablet_manager.cc:1428] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 00:49:19.537309 27596 tablet_bootstrap.cc:492] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676: No bootstrap required, opened a new log
I20260806 00:49:19.537397 27596 ts_tablet_manager.cc:1397] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 00:49:19.537708 27596 raft_consensus.cc:348] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.537796 27596 raft_consensus.cc:374] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:19.537817 27596 raft_consensus.cc:729] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4ee92563e1402997d02ba8b1fd8676, State: Initialized, Role: FOLLOWER
I20260806 00:49:19.537938 27596 consensus_queue.cc:260] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.538209 27596 ts_tablet_manager.cc:1428] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 00:49:19.538987 27602 tablet_bootstrap.cc:492] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5: No bootstrap required, opened a new log
I20260806 00:49:19.539078 27602 ts_tablet_manager.cc:1397] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 00:49:19.539520 27602 raft_consensus.cc:348] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.539652 27602 raft_consensus.cc:374] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:19.539690 27602 raft_consensus.cc:729] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6f3901094e34f41a3d4bdc5436cc8b5, State: Initialized, Role: FOLLOWER
I20260806 00:49:19.539844 27602 consensus_queue.cc:260] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.540117 27602 ts_tablet_manager.cc:1428] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 00:49:19.608218 27614 raft_consensus.cc:482] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:49:19.608383 27614 raft_consensus.cc:504] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.609543 27614 leader_election.cc:288] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a4ee92563e1402997d02ba8b1fd8676 (127.15.147.129:33443), f6f3901094e34f41a3d4bdc5436cc8b5 (127.15.147.130:39021)
I20260806 00:49:19.613544 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.609938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:33301 (local address 127.15.147.129:33443)
0806 00:49:19.610128 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:19.610135 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:19.610201 (+ 66us) server_negotiation.cc:407] Connection header received
0806 00:49:19.610396 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:19.610400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:19.610486 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:19.610594 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:19.610873 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.611350 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.612562 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.612972 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.613099 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.613158 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:19.613331 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 00:49:19.613408 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:49:19.613570 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.609816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:33443 (local address 127.15.147.131:33301)
0806 00:49:19.610105 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:19.610118 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:19.610263 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:19.610618 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:19.610621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:19.610640 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:19.610768 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.610773 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.611456 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.611461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.612438 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.612449 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.613100 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.613104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.613201 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.613222 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 00:49:19.613313 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 00:49:19.613389 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":2,"thread_start_us":99,"threads_started":1}
I20260806 00:49:19.613718 27588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.610472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:43219 (local address 127.15.147.130:39021)
0806 00:49:19.610610 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:19.610614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:19.610630 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:49:19.610688 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:19.610691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:19.610735 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:19.610812 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:19.611099 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.611581 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.612774 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.613165 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.613289 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.613342 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:19.613546 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 00:49:19.613613 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:49:19.613737 27711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.610279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:39021 (local address 127.15.147.131:43219)
0806 00:49:19.610513 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:19.610525 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:19.610601 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:19.610833 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:19.610836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:19.610851 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:19.610987 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.610991 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.611724 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.611727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.612658 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.612668 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.613289 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.613293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.613416 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.613434 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 00:49:19.613529 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 00:49:19.613594 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":24,"thread_start_us":90,"threads_started":1}
I20260806 00:49:19.614188 27098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50529b7a2cb74fb3abbd94ef2f2c8236" candidate_uuid: "c38adac127f84c8bbdb8a91ea391525e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4ee92563e1402997d02ba8b1fd8676" is_pre_election: true
I20260806 00:49:19.614270 27283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50529b7a2cb74fb3abbd94ef2f2c8236" candidate_uuid: "c38adac127f84c8bbdb8a91ea391525e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" is_pre_election: true
I20260806 00:49:19.614341 27098 raft_consensus.cc:2454] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c38adac127f84c8bbdb8a91ea391525e in term 0.
I20260806 00:49:19.614459 27283 raft_consensus.cc:2454] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c38adac127f84c8bbdb8a91ea391525e in term 0.
I20260806 00:49:19.614784 27396 leader_election.cc:302] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4ee92563e1402997d02ba8b1fd8676, c38adac127f84c8bbdb8a91ea391525e; no voters:
I20260806 00:49:19.615054 27614 raft_consensus.cc:2781] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:49:19.615139 27614 raft_consensus.cc:482] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:49:19.615195 27614 raft_consensus.cc:3037] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:19.616294 27614 raft_consensus.cc:504] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.616720 27614 leader_election.cc:288] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [CANDIDATE]: Term 1 election: Requested vote from peers 6a4ee92563e1402997d02ba8b1fd8676 (127.15.147.129:33443), f6f3901094e34f41a3d4bdc5436cc8b5 (127.15.147.130:39021)
I20260806 00:49:19.617148 27098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50529b7a2cb74fb3abbd94ef2f2c8236" candidate_uuid: "c38adac127f84c8bbdb8a91ea391525e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a4ee92563e1402997d02ba8b1fd8676"
I20260806 00:49:19.617172 27283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50529b7a2cb74fb3abbd94ef2f2c8236" candidate_uuid: "c38adac127f84c8bbdb8a91ea391525e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5"
I20260806 00:49:19.617280 27283 raft_consensus.cc:3037] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:19.617280 27098 raft_consensus.cc:3037] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:19.618398 27098 raft_consensus.cc:2454] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c38adac127f84c8bbdb8a91ea391525e in term 1.
I20260806 00:49:19.618398 27283 raft_consensus.cc:2454] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c38adac127f84c8bbdb8a91ea391525e in term 1.
I20260806 00:49:19.618690 27396 leader_election.cc:302] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a4ee92563e1402997d02ba8b1fd8676, c38adac127f84c8bbdb8a91ea391525e; no voters:
I20260806 00:49:19.618909 27614 raft_consensus.cc:2781] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:19.619091 27614 raft_consensus.cc:686] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [term 1 LEADER]: Becoming Leader. State: Replica: c38adac127f84c8bbdb8a91ea391525e, State: Running, Role: LEADER
I20260806 00:49:19.619333 27614 consensus_queue.cc:237] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } }
I20260806 00:49:19.621580 26866 catalog_manager.cc:5495] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e reported cstate change: term changed from 0 to 1, leader changed from <none> to c38adac127f84c8bbdb8a91ea391525e (127.15.147.131). New cstate: current_term: 1 leader_uuid: "c38adac127f84c8bbdb8a91ea391525e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c38adac127f84c8bbdb8a91ea391525e" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 35525 } health_report { overall_health: HEALTHY } } }
I20260806 00:49:19.650624 27593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.647045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39866 (local address 127.15.147.131:35525)
0806 00:49:19.647164 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:19.647169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:19.647221 (+ 52us) server_negotiation.cc:407] Connection header received
0806 00:49:19.647411 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:19.647414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:19.647471 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:19.647579 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:49:19.647827 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.648378 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.649190 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.649413 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.649517 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.649780 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:49:19.650243 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:49:19.650368 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:19.650428 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 00:49:19.650481 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:49:19.659893 27718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.656978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:33443 (local address 127.15.147.131:52275)
0806 00:49:19.657265 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:19.657277 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:19.657359 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:19.657565 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:19.657567 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:19.657573 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:49:19.657666 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.657669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.658281 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.658283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.658887 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.658895 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.659295 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.659298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.659364 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.659406 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:49:19.659644 (+ 238us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:49:19.659648 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 00:49:19.659697 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 00:49:19.659762 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":100,"threads_started":1}
I20260806 00:49:19.659883 27594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.657075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:52275 (local address 127.15.147.129:33443)
0806 00:49:19.657188 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:19.657192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:19.657336 (+ 144us) server_negotiation.cc:407] Connection header received
0806 00:49:19.657436 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:19.657438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:19.657483 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:19.657545 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:49:19.657746 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.658200 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.658981 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.659187 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.659258 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.659513 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:49:19.659576 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:49:19.659627 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:19.659702 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 00:49:19.659743 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 00:49:19.666121 27098 raft_consensus.cc:1261] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 FOLLOWER]: Refusing update from remote peer c38adac127f84c8bbdb8a91ea391525e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:19.666177 27283 raft_consensus.cc:1261] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 FOLLOWER]: Refusing update from remote peer c38adac127f84c8bbdb8a91ea391525e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:19.666734 27614 consensus_queue.cc:1035] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6f3901094e34f41a3d4bdc5436cc8b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:19.666940 27714 consensus_queue.cc:1035] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a4ee92563e1402997d02ba8b1fd8676" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:19.685051 27678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.681733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:35525 (local address 127.15.147.129:49429)
0806 00:49:19.681890 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:19.681906 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:19.681988 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:19.682228 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:19.682231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:19.682239 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 00:49:19.682348 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.682354 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.683204 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.683208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.683800 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:19.683809 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.684345 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.684348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:19.684436 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.684469 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 00:49:19.684758 (+ 289us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 00:49:19.684767 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 00:49:19.684832 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 00:49:19.684921 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 00:49:19.685051 27593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:19.681817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:49429 (local address 127.15.147.131:35525)
0806 00:49:19.681926 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:19.681931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:19.681946 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:49:19.682062 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:19.682065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:19.682120 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:19.682189 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:49:19.682458 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.683091 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.683929 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:19.684222 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:19.684343 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:19.684599 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:49:19.684676 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:49:19.684753 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:19.684840 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 00:49:19.684892 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 00:49:19.707377 26866 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:49:19.707558 26866 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:49:19.709601 26866 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 573d75e7bcd943bf900f0b7313c8bd35: Sending DeleteTablet for 3 replicas of tablet 50529b7a2cb74fb3abbd94ef2f2c8236
I20260806 00:49:19.710177 27066 tablet_service.cc:1514] Processing DeleteTablet for tablet 50529b7a2cb74fb3abbd94ef2f2c8236 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:19 UTC) from {username='oryx', principal='oryx/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:57214
I20260806 00:49:19.710265 27258 tablet_service.cc:1514] Processing DeleteTablet for tablet 50529b7a2cb74fb3abbd94ef2f2c8236 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:19 UTC) from {username='oryx', principal='oryx/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:55456
I20260806 00:49:19.710357 27636 tablet_replica.cc:330] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676: stopping tablet replica
I20260806 00:49:19.710460 27461 tablet_service.cc:1514] Processing DeleteTablet for tablet 50529b7a2cb74fb3abbd94ef2f2c8236 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:19 UTC) from {username='oryx', principal='oryx/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:39850
I20260806 00:49:19.710471 27636 raft_consensus.cc:2229] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:19.710588 27636 raft_consensus.cc:2258] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:19.710628 27634 tablet_replica.cc:330] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5: stopping tablet replica
I20260806 00:49:19.710757 27634 raft_consensus.cc:2229] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:19.710877 27634 raft_consensus.cc:2258] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:19.711038 27632 tablet_replica.cc:330] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e: stopping tablet replica
I20260806 00:49:19.711146 27632 raft_consensus.cc:2229] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:49:19.711458 27632 raft_consensus.cc:2258] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:19.711740 27636 ts_tablet_manager.cc:1905] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:19.712182 27632 ts_tablet_manager.cc:1905] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:19.712209 27634 ts_tablet_manager.cc:1905] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:19.714520 27636 ts_tablet_manager.cc:1918] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:49:19.714689 27636 log.cc:1196] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-0/wal/wals/50529b7a2cb74fb3abbd94ef2f2c8236
I20260806 00:49:19.714715 27634 ts_tablet_manager.cc:1918] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:49:19.714783 27634 log.cc:1196] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/50529b7a2cb74fb3abbd94ef2f2c8236
I20260806 00:49:19.714829 27632 ts_tablet_manager.cc:1918] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 00:49:19.714918 27632 log.cc:1196] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/50529b7a2cb74fb3abbd94ef2f2c8236
I20260806 00:49:19.715049 27634 ts_tablet_manager.cc:1939] T 50529b7a2cb74fb3abbd94ef2f2c8236 P f6f3901094e34f41a3d4bdc5436cc8b5: Deleting consensus metadata
I20260806 00:49:19.715201 27632 ts_tablet_manager.cc:1939] T 50529b7a2cb74fb3abbd94ef2f2c8236 P c38adac127f84c8bbdb8a91ea391525e: Deleting consensus metadata
I20260806 00:49:19.715231 27636 ts_tablet_manager.cc:1939] T 50529b7a2cb74fb3abbd94ef2f2c8236 P 6a4ee92563e1402997d02ba8b1fd8676: Deleting consensus metadata
I20260806 00:49:19.715821 26852 catalog_manager.cc:4841] TS f6f3901094e34f41a3d4bdc5436cc8b5 (127.15.147.130:39021): tablet 50529b7a2cb74fb3abbd94ef2f2c8236 (table test-table [id=bf7628c0b91d4fcaa433bb3d8909a7b3]) successfully deleted
I20260806 00:49:19.716003 26852 catalog_manager.cc:4841] TS 6a4ee92563e1402997d02ba8b1fd8676 (127.15.147.129:33443): tablet 50529b7a2cb74fb3abbd94ef2f2c8236 (table test-table [id=bf7628c0b91d4fcaa433bb3d8909a7b3]) successfully deleted
I20260806 00:49:19.716252 26850 catalog_manager.cc:4841] TS c38adac127f84c8bbdb8a91ea391525e (127.15.147.131:35525): tablet 50529b7a2cb74fb3abbd94ef2f2c8236 (table test-table [id=bf7628c0b91d4fcaa433bb3d8909a7b3]) successfully deleted
I20260806 00:49:19.716965 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 26974
I20260806 00:49:19.734845 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 27171
I20260806 00:49:19.758416 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 27370
I20260806 00:49:19.770666 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 26829
2026-08-06T00:49:19Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3369 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:49:19 dist-test-slave-45dx krb5kdc[27731](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:49:19 dist-test-slave-45dx krb5kdc[27731](info): set up 2 sockets
Aug 06 00:49:19 dist-test-slave-45dx krb5kdc[27731](info): commencing operation
krb5kdc: starting...
W20260806 00:49:21.009604 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:49:21 dist-test-slave-45dx krb5kdc[27731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:49:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:49:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:49:21.151703 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:42817
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:32815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:42817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:21.266918 27753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:21.267246 27753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:21.267333 27753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:21.272080 27753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:49:21.272209 27753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:21.272255 27753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:21.272339 27753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:49:21.272383 27753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:49:21.278535 27753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:32815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:42817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:42817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:21.279718 27753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:21.280889 27753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:21.289857 27768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:21.289933 27761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:21.289881 27765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:21.290071 27753 server_base.cc:1034] running on GCE node
I20260806 00:49:21.290843 27753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:21.292030 27753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:21.293236 27753 hybrid_clock.cc:648] HybridClock initialized: now 1785977361293209 us; error 46 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T00:49:21Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1533 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:49:21 dist-test-slave-45dx krb5kdc[27773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:49:21 dist-test-slave-45dx krb5kdc[27773](info): set up 2 sockets
Aug 06 00:49:21 dist-test-slave-45dx krb5kdc[27773](info): commencing operation
krb5kdc: starting...
W20260806 00:49:22.520632 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:49:22 dist-test-slave-45dx krb5kdc[27773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:49:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:49:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:49:22.654508 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:45281
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:45281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:22.768543 27792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:22.768824 27792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:22.768867 27792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:22.772943 27792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:49:22.773010 27792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:22.773029 27792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:22.773046 27792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:49:22.773061 27792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:49:22.778805 27792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:45281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:45281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:22.780028 27792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:22.781255 27792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:49:22.790488 27792 server_base.cc:1034] running on GCE node
W20260806 00:49:22.790439 27798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:22.790519 27799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:22.790439 27801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:22.791388 27792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:22.792546 27792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:22.793735 27792 hybrid_clock.cc:648] HybridClock initialized: now 1785977362793718 us; error 29 us; skew 500 ppm
Aug 06 00:49:22 dist-test-slave-45dx krb5kdc[27773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977362, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:22.796762 27792 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:49:22.797678 27792 webserver.cc:460] Webserver started at http://127.15.147.190:41569/ using document root <none> and password file <none>
I20260806 00:49:22.798236 27792 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:22.798327 27792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:22.798563 27792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:22.801085 27792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "765bd69a124d4047a3e56e32ab65db43"
format_stamp: "Formatted at 2026-08-06 00:49:22 on dist-test-slave-45dx"
server_key: "7360afb8df9fdd354bf826904f48d2ad"
server_key_iv: "1440fa7eebc3741ba74ae55865bbb55c"
server_key_version: "encryptionkey@0"
I20260806 00:49:22.801786 27792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "765bd69a124d4047a3e56e32ab65db43"
format_stamp: "Formatted at 2026-08-06 00:49:22 on dist-test-slave-45dx"
server_key: "7360afb8df9fdd354bf826904f48d2ad"
server_key_iv: "1440fa7eebc3741ba74ae55865bbb55c"
server_key_version: "encryptionkey@0"
I20260806 00:49:22.805912 27792 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:49:22.808455 27810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:22.809795 27792 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:49:22.809979 27792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "765bd69a124d4047a3e56e32ab65db43"
format_stamp: "Formatted at 2026-08-06 00:49:22 on dist-test-slave-45dx"
server_key: "7360afb8df9fdd354bf826904f48d2ad"
server_key_iv: "1440fa7eebc3741ba74ae55865bbb55c"
server_key_version: "encryptionkey@0"
I20260806 00:49:22.810104 27792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:22.832677 27792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:22.836339 27792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:22.836572 27792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:22.844169 27792 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:45281
I20260806 00:49:22.844198 27882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:45281 every 8 connection(s)
I20260806 00:49:22.845238 27792 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:49:22.848026 27884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:22.852119 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 27792
I20260806 00:49:22.852265 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:49:22.852625 15950 external_mini_cluster.cc:1428] Setting key 594a8592f5b5f71f61d20cba6562f887
I20260806 00:49:22.854477 27884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43: Bootstrap starting.
I20260806 00:49:22.856818 27884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:22.857589 27884 log.cc:824] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:22.859793 27884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43: No bootstrap required, opened a new log
Aug 06 00:49:22 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:49:22.864876 27884 raft_consensus.cc:348] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765bd69a124d4047a3e56e32ab65db43" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 45281 } }
I20260806 00:49:22.865181 27884 raft_consensus.cc:374] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:22.865249 27884 raft_consensus.cc:729] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 765bd69a124d4047a3e56e32ab65db43, State: Initialized, Role: FOLLOWER
I20260806 00:49:22.865769 27884 consensus_queue.cc:260] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765bd69a124d4047a3e56e32ab65db43" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 45281 } }
I20260806 00:49:22.865949 27884 raft_consensus.cc:388] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:49:22.866021 27884 raft_consensus.cc:482] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:49:22.866191 27884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:22.867254 27884 raft_consensus.cc:504] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765bd69a124d4047a3e56e32ab65db43" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 45281 } }
I20260806 00:49:22.867671 27884 leader_election.cc:302] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 765bd69a124d4047a3e56e32ab65db43; no voters:
I20260806 00:49:22.868049 27884 leader_election.cc:288] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:49:22.868139 27890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:22.868371 27890 raft_consensus.cc:686] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [term 1 LEADER]: Becoming Leader. State: Replica: 765bd69a124d4047a3e56e32ab65db43, State: Running, Role: LEADER
I20260806 00:49:22.868595 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:22.854162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45496 (local address 127.15.147.190:45281)
0806 00:49:22.854662 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:22.854679 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:22.854716 (+ 37us) server_negotiation.cc:407] Connection header received
0806 00:49:22.855361 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:22.855378 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:22.855713 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:22.856059 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:22.856924 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:22.858036 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:22.859278 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:22.859754 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:22.859898 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:22.862791 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:22.862801 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:22.862822 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:22.862862 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:22.864978 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:22.865524 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:22.865530 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:22.865536 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:22.865677 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:22.865972 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:22.865977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:22.865979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:22.866550 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:22.866808 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:22.867130 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 00:49:22.867504 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":136,"threads_started":1}
I20260806 00:49:22.869526 27884 sys_catalog.cc:564] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:49:22.868703 27890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765bd69a124d4047a3e56e32ab65db43" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 45281 } }
I20260806 00:49:22.872529 27892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 765bd69a124d4047a3e56e32ab65db43. Latest consensus state: current_term: 1 leader_uuid: "765bd69a124d4047a3e56e32ab65db43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765bd69a124d4047a3e56e32ab65db43" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 45281 } } }
I20260806 00:49:22.872771 27892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:22.872713 27891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "765bd69a124d4047a3e56e32ab65db43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765bd69a124d4047a3e56e32ab65db43" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 45281 } } }
I20260806 00:49:22.873238 27891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [sys.catalog]: This master's current role is: LEADER
W20260806 00:49:22.875195 27909 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:49:22.875291 27909 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:49:22.875371 27910 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:49:22.876103 27910 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:49:22.882081 27910 catalog_manager.cc:1347] Generated new cluster ID: 9c55fd22763e400c94efdb3db72c95cb
I20260806 00:49:22.882154 27910 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:49:22.894541 27910 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:49:22.895780 27910 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:49:22.905617 27910 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43: Generated new TSK 0
I20260806 00:49:22.906348 27910 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
I20260806 00:49:22.940109 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:45281
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:23.059753 27920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:23.060055 27920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:23.060101 27920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:23.064250 27920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:23.064399 27920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:23.064467 27920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:49:23.070194 27920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:45281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:23.071465 27920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:23.072546 27920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:23.082616 27933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:23.082682 27920 server_base.cc:1034] running on GCE node
W20260806 00:49:23.082616 27929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:23.082619 27928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:23.083362 27920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:23.084120 27920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:23.085328 27920 hybrid_clock.cc:648] HybridClock initialized: now 1785977363085302 us; error 34 us; skew 500 ppm
Aug 06 00:49:23 dist-test-slave-45dx krb5kdc[27773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977363, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:23.088222 27920 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:49:23.089193 27920 webserver.cc:460] Webserver started at http://127.15.147.129:45817/ using document root <none> and password file <none>
I20260806 00:49:23.089735 27920 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:23.089807 27920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:23.089969 27920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:23.091918 27920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "36e3bc3c8af0468d812257e46b352c1a"
format_stamp: "Formatted at 2026-08-06 00:49:23 on dist-test-slave-45dx"
server_key: "be0d87d497dab5846e175cd47e96a330"
server_key_iv: "7c236448ea55a9c1f6dff8ef12e80fd8"
server_key_version: "encryptionkey@0"
I20260806 00:49:23.092479 27920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "36e3bc3c8af0468d812257e46b352c1a"
format_stamp: "Formatted at 2026-08-06 00:49:23 on dist-test-slave-45dx"
server_key: "be0d87d497dab5846e175cd47e96a330"
server_key_iv: "7c236448ea55a9c1f6dff8ef12e80fd8"
server_key_version: "encryptionkey@0"
I20260806 00:49:23.096544 27920 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 00:49:23.099028 27949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:23.100180 27920 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:49:23.100289 27920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "36e3bc3c8af0468d812257e46b352c1a"
format_stamp: "Formatted at 2026-08-06 00:49:23 on dist-test-slave-45dx"
server_key: "be0d87d497dab5846e175cd47e96a330"
server_key_iv: "7c236448ea55a9c1f6dff8ef12e80fd8"
server_key_version: "encryptionkey@0"
I20260806 00:49:23.100428 27920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:23.125443 27920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:23.128480 27920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:23.128633 27920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:23.129212 27920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:23.129980 27920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:23.130028 27920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:23.130067 27920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:23.130087 27920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:23.138991 27920 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:43709
I20260806 00:49:23.139038 28113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:43709 every 8 connection(s)
I20260806 00:49:23.139890 27920 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:49:23.148438 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 27920
I20260806 00:49:23.148597 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:49:23.148968 15950 external_mini_cluster.cc:1428] Setting key 9427adfebdf09fae443d76fe54bc891a
Aug 06 00:49:23 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977363, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:49:23.153880 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:23.141634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:59585 (local address 127.15.147.190:45281)
0806 00:49:23.141773 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:23.141776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:23.142877 (+ 1101us) server_negotiation.cc:407] Connection header received
0806 00:49:23.143705 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:23.143708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:23.143751 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:23.143824 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:23.145092 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:23.145643 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:23.146791 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:23.147026 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:23.147113 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:23.149825 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:23.149831 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:23.149833 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:23.149883 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:23.151382 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:23.151903 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:23.151907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:23.151908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:23.152028 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:23.152519 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:23.152525 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:23.152528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:23.152801 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:23.152890 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:23.153636 (+ 746us) server_negotiation.cc:299] Negotiation successful
0806 00:49:23.153750 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:49:23.155165 28117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:23.142031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:59585)
0806 00:49:23.142708 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:23.142749 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:23.143534 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:23.143972 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:23.143978 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:23.144478 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:23.144952 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:23.144965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:23.145783 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:23.145786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:23.146663 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:23.146673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:23.147190 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:23.147193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:23.147365 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:23.147991 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:23.148008 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:23.149660 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:23.151526 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:23.151531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:23.151545 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:23.151803 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:23.152137 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:23.152140 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:23.152142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:23.152390 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:23.152984 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:23.152992 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:23.153327 (+ 335us) client_negotiation.cc:769] Sending connection context
0806 00:49:23.153559 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 00:49:23.154082 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":104,"threads_started":1}
I20260806 00:49:23.156443 28115 heartbeater.cc:347] Connected to a master server at 127.15.147.190:45281
I20260806 00:49:23.156692 28115 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:23.157174 28115 heartbeater.cc:511] Master 127.15.147.190:45281 requested a full tablet report, sending...
I20260806 00:49:23.158895 27834 ts_manager.cc:194] Registered new tserver with Master: 36e3bc3c8af0468d812257e46b352c1a (127.15.147.129:43709)
I20260806 00:49:23.160095 27834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:59585
WARNING: no policy specified for kudu/127.15.147.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
I20260806 00:49:23.179566 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:45281
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:23.289865 28123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:23.290096 28123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:23.290169 28123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:23.294064 28123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:23.294109 28123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:23.294157 28123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:49:23.299140 28123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:45281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:23.300115 28123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:23.301115 28123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:23.310678 28133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:23.310662 28136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:23.310791 28138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:23.310897 28123 server_base.cc:1034] running on GCE node
I20260806 00:49:23.311406 28123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:23.312000 28123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:23.313179 28123 hybrid_clock.cc:648] HybridClock initialized: now 1785977363313128 us; error 63 us; skew 500 ppm
Aug 06 00:49:23 dist-test-slave-45dx krb5kdc[27773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977363, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:23.315861 28123 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:49:23.316735 28123 webserver.cc:460] Webserver started at http://127.15.147.130:32835/ using document root <none> and password file <none>
I20260806 00:49:23.317270 28123 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:23.317370 28123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:23.317585 28123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:23.319532 28123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "f9d7e18488ff46b3a48da4a7302481da"
format_stamp: "Formatted at 2026-08-06 00:49:23 on dist-test-slave-45dx"
server_key: "1040701fa4ab3190722b88d815dee621"
server_key_iv: "8e6de7fa8e6314e3efca0b9771f9308a"
server_key_version: "encryptionkey@0"
I20260806 00:49:23.320103 28123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "f9d7e18488ff46b3a48da4a7302481da"
format_stamp: "Formatted at 2026-08-06 00:49:23 on dist-test-slave-45dx"
server_key: "1040701fa4ab3190722b88d815dee621"
server_key_iv: "8e6de7fa8e6314e3efca0b9771f9308a"
server_key_version: "encryptionkey@0"
I20260806 00:49:23.323818 28123 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:49:23.326313 28152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:23.327417 28123 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:49:23.327550 28123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "f9d7e18488ff46b3a48da4a7302481da"
format_stamp: "Formatted at 2026-08-06 00:49:23 on dist-test-slave-45dx"
server_key: "1040701fa4ab3190722b88d815dee621"
server_key_iv: "8e6de7fa8e6314e3efca0b9771f9308a"
server_key_version: "encryptionkey@0"
I20260806 00:49:23.327656 28123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:23.358681 28123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:23.361639 28123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:23.361850 28123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:23.362447 28123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:23.363260 28123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:23.363327 28123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:23.363413 28123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:23.363453 28123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:23.372841 28123 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:45285
I20260806 00:49:23.372889 28306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:45285 every 8 connection(s)
I20260806 00:49:23.373834 28123 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:49:23.376528 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 28123
I20260806 00:49:23.376655 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:49:23.377008 15950 external_mini_cluster.cc:1428] Setting key 3a6a5a358e811bba5801a2f23ff4cc0b
Aug 06 00:49:23 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977363, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.147.131@KRBTEST.COM; defaulting to no policy
I20260806 00:49:23.388447 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:23.375579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:56825 (local address 127.15.147.190:45281)
0806 00:49:23.375710 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:23.375714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:23.376983 (+ 1269us) server_negotiation.cc:407] Connection header received
0806 00:49:23.378050 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:23.378054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:23.378099 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:23.378186 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:23.379746 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:23.380276 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:23.381999 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:23.382228 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:23.382311 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:23.384826 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:23.384829 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:23.384832 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:23.384874 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:23.386313 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:23.386936 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:23.386938 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:23.386939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:23.387016 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:23.387371 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:23.387377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:23.387381 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:23.387559 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:23.387664 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:23.388120 (+ 456us) server_negotiation.cc:299] Negotiation successful
0806 00:49:23.388236 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:49:23.389221 28313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:23.375944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:56825)
0806 00:49:23.376759 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:23.376818 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:23.377845 (+ 1027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:23.378384 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:23.378395 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:23.378922 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:23.379597 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:23.379620 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:23.380432 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:23.380436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:23.381864 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:23.381880 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:23.382389 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:23.382393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:23.382576 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:23.383187 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:23.383202 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:23.384647 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:23.386430 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:23.386436 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:23.386450 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:23.386836 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:23.387115 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:23.387119 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:23.387121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:23.387265 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:23.387665 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:23.387673 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:23.387896 (+ 223us) client_negotiation.cc:769] Sending connection context
0806 00:49:23.388083 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 00:49:23.388423 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":117,"threads_started":1}
I20260806 00:49:23.390482 28307 heartbeater.cc:347] Connected to a master server at 127.15.147.190:45281
I20260806 00:49:23.390713 28307 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.131@KRBTEST.COM" created.
I20260806 00:49:23.391217 28307 heartbeater.cc:511] Master 127.15.147.190:45281 requested a full tablet report, sending...
I20260806 00:49:23.392283 27834 ts_manager.cc:194] Registered new tserver with Master: f9d7e18488ff46b3a48da4a7302481da (127.15.147.130:45285)
I20260806 00:49:23.393137 27834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:56825
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
I20260806 00:49:23.406736 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:45281
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:23.518896 28317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:23.519150 28317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:23.519224 28317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:23.523273 28317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:23.523327 28317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:23.523432 28317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:49:23.528640 28317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:45281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:23.529645 28317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:23.530701 28317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:23.540540 28326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:23.540561 28323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:23.540668 28317 server_base.cc:1034] running on GCE node
W20260806 00:49:23.540540 28324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:23.541247 28317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:23.541939 28317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:23.543118 28317 hybrid_clock.cc:648] HybridClock initialized: now 1785977363543100 us; error 29 us; skew 500 ppm
Aug 06 00:49:23 dist-test-slave-45dx krb5kdc[27773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977363, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:23.545912 28317 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:49:23.546756 28317 webserver.cc:460] Webserver started at http://127.15.147.131:38611/ using document root <none> and password file <none>
I20260806 00:49:23.547273 28317 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:23.547331 28317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:23.547493 28317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:23.549484 28317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "0e1dffdcd7e946bea1e4ad0a1db6c69e"
format_stamp: "Formatted at 2026-08-06 00:49:23 on dist-test-slave-45dx"
server_key: "f7ce083f7bb9f0d885d5f6d96e1cb4ec"
server_key_iv: "5e0436b141955cdf947e708d96a1ffed"
server_key_version: "encryptionkey@0"
I20260806 00:49:23.550037 28317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "0e1dffdcd7e946bea1e4ad0a1db6c69e"
format_stamp: "Formatted at 2026-08-06 00:49:23 on dist-test-slave-45dx"
server_key: "f7ce083f7bb9f0d885d5f6d96e1cb4ec"
server_key_iv: "5e0436b141955cdf947e708d96a1ffed"
server_key_version: "encryptionkey@0"
I20260806 00:49:23.553586 28317 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:49:23.555799 28337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:23.556983 28317 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:49:23.557077 28317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "0e1dffdcd7e946bea1e4ad0a1db6c69e"
format_stamp: "Formatted at 2026-08-06 00:49:23 on dist-test-slave-45dx"
server_key: "f7ce083f7bb9f0d885d5f6d96e1cb4ec"
server_key_iv: "5e0436b141955cdf947e708d96a1ffed"
server_key_version: "encryptionkey@0"
I20260806 00:49:23.557157 28317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:23.573159 28317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:23.576087 28317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:23.576244 28317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:23.576857 28317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:23.577661 28317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:23.577708 28317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:23.577749 28317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:23.577764 28317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:23.586918 28317 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:37985
I20260806 00:49:23.586943 28491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:37985 every 8 connection(s)
I20260806 00:49:23.587878 28317 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:49:23.592939 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 28317
I20260806 00:49:23.593087 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:49:23.593425 15950 external_mini_cluster.cc:1428] Setting key dde422155193daf2afffdcf344369ec6
Aug 06 00:49:23 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977363, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:49:23.601927 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:23.589727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:57193 (local address 127.15.147.190:45281)
0806 00:49:23.589903 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:23.589907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:23.590898 (+ 991us) server_negotiation.cc:407] Connection header received
0806 00:49:23.591809 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:23.591813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:23.591882 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:23.591993 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:23.593489 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:23.594197 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:23.595747 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:23.595974 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:23.596059 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:23.598548 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:23.598552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:23.598555 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:23.598586 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:23.599837 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:23.600389 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:23.600392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:23.600394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:23.600486 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:23.600830 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:23.600833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:23.600835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:23.601019 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:23.601123 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:23.601616 (+ 493us) server_negotiation.cc:299] Negotiation successful
0806 00:49:23.601740 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 00:49:23.602720 28503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:23.590091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.131:57193)
0806 00:49:23.590741 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:23.590781 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:23.591597 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:23.592183 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:23.592189 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:23.592593 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:23.593306 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:23.593328 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:23.594342 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:23.594348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:23.595628 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:23.595641 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:23.596113 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:23.596116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:23.596269 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:23.596885 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:23.596902 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:23.598366 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:23.599957 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:23.599963 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:23.599978 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:23.600239 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:23.600580 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:23.600583 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:23.600585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:23.600728 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:23.601122 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:23.601129 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:23.601376 (+ 247us) client_negotiation.cc:769] Sending connection context
0806 00:49:23.601557 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 00:49:23.601909 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":110,"threads_started":1}
I20260806 00:49:23.603801 28497 heartbeater.cc:347] Connected to a master server at 127.15.147.190:45281
I20260806 00:49:23.604104 28497 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:23.604672 28497 heartbeater.cc:511] Master 127.15.147.190:45281 requested a full tablet report, sending...
I20260806 00:49:23.605746 27834 ts_manager.cc:194] Registered new tserver with Master: 0e1dffdcd7e946bea1e4ad0a1db6c69e (127.15.147.131:37985)
I20260806 00:49:23.606305 27834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:57193
I20260806 00:49:23.607362 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.129.keytab.
Entry for principal oryx/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.129.keytab.
WARNING: no policy specified for oryx/127.15.147.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.147.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.130.keytab.
Entry for principal oryx/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.130.keytab.
WARNING: no policy specified for oryx/127.15.147.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.147.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.131.keytab.
Entry for principal oryx/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.131.keytab.
I20260806 00:49:23.686911 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 27920
I20260806 00:49:23.706568 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 28123
I20260806 00:49:23.720667 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 28317
I20260806 00:49:23.730221 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 27792
I20260806 00:49:23.739562 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:45281
--webserver_interface=127.15.147.190
--webserver_port=41569
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:45281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:23.861043 28510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:23.861371 28510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:23.861418 28510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:23.865958 28510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:49:23.866026 28510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:23.866045 28510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:23.866063 28510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:49:23.866117 28510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:49:23.871644 28510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:45281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:45281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=41569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:23.872804 28510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:23.873907 28510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:23.883224 28516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:23.883203 28515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:23.883280 28520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:23.883226 28510 server_base.cc:1034] running on GCE node
I20260806 00:49:23.884138 28510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:23.885344 28510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:23.886555 28510 hybrid_clock.cc:648] HybridClock initialized: now 1785977363886539 us; error 29 us; skew 500 ppm
Aug 06 00:49:23 dist-test-slave-45dx krb5kdc[27773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977363, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:23.889660 28510 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:49:23.890684 28510 webserver.cc:460] Webserver started at http://127.15.147.190:41569/ using document root <none> and password file <none>
I20260806 00:49:23.891304 28510 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:23.891426 28510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:23.895529 28510 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:49:23.897914 28531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:23.899166 28510 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:49:23.899312 28510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "765bd69a124d4047a3e56e32ab65db43"
format_stamp: "Formatted at 2026-08-06 00:49:22 on dist-test-slave-45dx"
server_key: "7360afb8df9fdd354bf826904f48d2ad"
server_key_iv: "1440fa7eebc3741ba74ae55865bbb55c"
server_key_version: "encryptionkey@0"
I20260806 00:49:23.899876 28510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:23.922837 28510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:23.926308 28510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:23.926544 28510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:23.934219 28510 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:45281
I20260806 00:49:23.934229 28611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:45281 every 8 connection(s)
I20260806 00:49:23.935321 28510 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:49:23.936982 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 28510
I20260806 00:49:23.937471 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:43709
--local_ip_for_outbound_sockets=127.15.147.129
--tserver_master_addrs=127.15.147.190:45281
--webserver_port=45817
--webserver_interface=127.15.147.129
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.147.129
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.129.keytab with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:49:23.939826 28612 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:49:23.941534 28612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43: Bootstrap starting.
I20260806 00:49:23.959967 28612 log.cc:824] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:23.966950 28612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:49:23.967387 28612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43: Bootstrap complete.
I20260806 00:49:23.971994 28612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765bd69a124d4047a3e56e32ab65db43" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 45281 } }
I20260806 00:49:23.972545 28612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 765bd69a124d4047a3e56e32ab65db43, State: Initialized, Role: FOLLOWER
I20260806 00:49:23.973122 28612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765bd69a124d4047a3e56e32ab65db43" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 45281 } }
I20260806 00:49:23.973248 28612 raft_consensus.cc:388] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:49:23.973335 28612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:49:23.973433 28612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [term 1 FOLLOWER]: Advancing to term 2
I20260806 00:49:23.975471 28612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765bd69a124d4047a3e56e32ab65db43" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 45281 } }
I20260806 00:49:23.975823 28612 leader_election.cc:302] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 765bd69a124d4047a3e56e32ab65db43; no voters:
I20260806 00:49:23.976161 28612 leader_election.cc:288] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 00:49:23.976362 28622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 00:49:23.976744 28622 raft_consensus.cc:686] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [term 2 LEADER]: Becoming Leader. State: Replica: 765bd69a124d4047a3e56e32ab65db43, State: Running, Role: LEADER
I20260806 00:49:23.977119 28622 consensus_queue.cc:237] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765bd69a124d4047a3e56e32ab65db43" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 45281 } }
I20260806 00:49:23.977437 28612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:49:23.978778 28623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "765bd69a124d4047a3e56e32ab65db43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765bd69a124d4047a3e56e32ab65db43" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 45281 } } }
I20260806 00:49:23.978816 28625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 765bd69a124d4047a3e56e32ab65db43. Latest consensus state: current_term: 2 leader_uuid: "765bd69a124d4047a3e56e32ab65db43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "765bd69a124d4047a3e56e32ab65db43" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 45281 } } }
I20260806 00:49:23.978900 28623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:23.978922 28625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:23.979336 28634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:49:23.982903 28634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:49:23.983686 28634 catalog_manager.cc:1259] Loaded cluster ID: 9c55fd22763e400c94efdb3db72c95cb
I20260806 00:49:23.983796 28634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:49:23.986581 28634 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:49:23.987418 28634 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 765bd69a124d4047a3e56e32ab65db43: Loaded TSK: 0
I20260806 00:49:23.988813 28634 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:49:24.062592 28615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:24.062943 28615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:24.063038 28615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:24.067274 28615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:24.067361 28615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:24.067451 28615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:49:24.072786 28615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.129.keytab
--principal=oryx/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:43709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=45817
--tserver_master_addrs=127.15.147.190:45281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:24.073863 28615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:24.074926 28615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:24.085153 28651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:24.085179 28650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:24.085429 28615 server_base.cc:1034] running on GCE node
W20260806 00:49:24.085197 28653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:24.085999 28615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:24.086666 28615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:24.087847 28615 hybrid_clock.cc:648] HybridClock initialized: now 1785977364087822 us; error 45 us; skew 500 ppm
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977364, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:24.090965 28615 init.cc:377] Logged in from keytab as oryx/127.15.147.129@KRBTEST.COM (short username oryx)
I20260806 00:49:24.091868 28615 webserver.cc:460] Webserver started at http://127.15.147.129:45817/ using document root <none> and password file <none>
I20260806 00:49:24.092507 28615 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:24.092612 28615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:24.096764 28615 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:49:24.099005 28666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:24.100409 28615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:49:24.100534 28615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "36e3bc3c8af0468d812257e46b352c1a"
format_stamp: "Formatted at 2026-08-06 00:49:23 on dist-test-slave-45dx"
server_key: "be0d87d497dab5846e175cd47e96a330"
server_key_iv: "7c236448ea55a9c1f6dff8ef12e80fd8"
server_key_version: "encryptionkey@0"
I20260806 00:49:24.101089 28615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:24.133980 28615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:24.137499 28615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:24.137722 28615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:24.138370 28615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:24.139246 28615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:24.139317 28615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:24.139410 28615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:24.139448 28615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:24.148999 28615 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:43709
I20260806 00:49:24.149024 28836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:43709 every 8 connection(s)
I20260806 00:49:24.150204 28615 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:49:24.155898 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 28615
I20260806 00:49:24.156466 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:45285
--local_ip_for_outbound_sockets=127.15.147.130
--tserver_master_addrs=127.15.147.190:45281
--webserver_port=32835
--webserver_interface=127.15.147.130
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.147.130
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.130.keytab with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:24.164388 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:24.152544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:41755)
0806 00:49:24.153271 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:24.153326 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:24.154137 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:24.155151 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:24.155157 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:24.155526 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:24.156017 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:24.156039 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.157937 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.157941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:24.158796 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:24.158806 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.159599 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.159604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:24.159802 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.160480 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:24.160507 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:24.163353 (+ 2846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":149,"threads_started":1}
W20260806 00:49:24.164390 28839 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:24.152450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:41755 (local address 127.15.147.190:45281)
0806 00:49:24.152906 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:24.152922 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:24.153479 (+ 557us) server_negotiation.cc:407] Connection header received
0806 00:49:24.154505 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:24.154517 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:24.154795 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:24.155070 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:24.156205 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.157793 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.158947 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.159427 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.159610 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.163591 (+ 3981us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.129:41755: BlockingRecv error: recv got EOF from 127.15.147.129:41755 (error 108)
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":122,"threads_started":1}
W20260806 00:49:24.165069 28837 heartbeater.cc:644] Failed to heartbeat to 127.15.147.190:45281 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.147.190:45281: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:24.170655 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:24.165549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:35073)
0806 00:49:24.165664 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:24.165679 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:24.165782 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:24.166036 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:24.166039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:24.166206 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:24.166297 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:24.166302 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.167026 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.167031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:24.167584 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:24.167592 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.168040 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.168043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:24.168141 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.168667 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:24.168679 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:24.170519 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":23}
W20260806 00:49:24.170889 28839 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:24.165679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:35073 (local address 127.15.147.190:45281)
0806 00:49:24.165818 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:24.165824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:24.165841 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:49:24.165890 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:24.165892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:24.165932 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:24.166019 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:24.166420 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.166916 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.167704 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.167929 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.168036 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.170647 (+ 2611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.129:35073: BlockingRecv error: recv got EOF from 127.15.147.129:35073 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:24.176050 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:24.171254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:54693)
0806 00:49:24.171377 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:24.171392 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:24.171469 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:24.171695 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:24.171699 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:24.171849 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:24.171927 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:24.171932 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.172684 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.172687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:24.173206 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:24.173213 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.173660 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.173663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:24.173743 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.174218 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:24.174231 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:24.175923 (+ 1692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 00:49:24.176275 28839 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:24.171361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:54693 (local address 127.15.147.190:45281)
0806 00:49:24.171462 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:24.171465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:24.171477 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:49:24.171548 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:24.171550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:24.171586 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:24.171677 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:24.172039 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.172577 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.173328 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.173547 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.173655 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.176041 (+ 2386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.129:54693: BlockingRecv error: recv got EOF from 127.15.147.129:54693 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260806 00:49:24.176376 28837 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:49:24.282588 28841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:24.282837 28841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:24.282914 28841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:24.287032 28841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:24.287099 28841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:24.287199 28841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:49:24.293133 28841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.130.keytab
--principal=oryx/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:45285
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=32835
--tserver_master_addrs=127.15.147.190:45281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:24.294350 28841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:24.295428 28841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:24.305400 28853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:24.305413 28848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:24.305517 28847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:24.305718 28841 server_base.cc:1034] running on GCE node
I20260806 00:49:24.306126 28841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:24.306780 28841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:24.307972 28841 hybrid_clock.cc:648] HybridClock initialized: now 1785977364307959 us; error 32 us; skew 500 ppm
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977364, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:24.310817 28841 init.cc:377] Logged in from keytab as oryx/127.15.147.130@KRBTEST.COM (short username oryx)
I20260806 00:49:24.311676 28841 webserver.cc:460] Webserver started at http://127.15.147.130:32835/ using document root <none> and password file <none>
I20260806 00:49:24.312266 28841 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:24.312407 28841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:24.316185 28841 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 00:49:24.318298 28863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:24.319485 28841 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:49:24.319620 28841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "f9d7e18488ff46b3a48da4a7302481da"
format_stamp: "Formatted at 2026-08-06 00:49:23 on dist-test-slave-45dx"
server_key: "1040701fa4ab3190722b88d815dee621"
server_key_iv: "8e6de7fa8e6314e3efca0b9771f9308a"
server_key_version: "encryptionkey@0"
I20260806 00:49:24.320189 28841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:24.349856 28841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:24.353125 28841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:24.353364 28841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:24.354033 28841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:24.354928 28841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:24.355010 28841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:24.355083 28841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:24.355126 28841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:24.364063 28841 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:45285
I20260806 00:49:24.364099 29038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:45285 every 8 connection(s)
I20260806 00:49:24.365157 28841 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:24.375200 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 28841
I20260806 00:49:24.375658 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:37985
--local_ip_for_outbound_sockets=127.15.147.131
--tserver_master_addrs=127.15.147.190:45281
--webserver_port=38611
--webserver_interface=127.15.147.131
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:38769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.147.131
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.131.keytab with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:24.376644 28839 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:24.366997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:57493 (local address 127.15.147.190:45281)
0806 00:49:24.367138 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:24.367141 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:24.368226 (+ 1085us) server_negotiation.cc:407] Connection header received
0806 00:49:24.369169 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:24.369172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:24.369238 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:24.369348 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:24.370542 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.371072 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.372230 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.372517 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.372633 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.376453 (+ 3820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.130:57493: BlockingRecv error: recv got EOF from 127.15.147.130:57493 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 00:49:24.377225 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:24.367406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:57493)
0806 00:49:24.368076 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:24.368118 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:24.368959 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:24.369540 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:24.369547 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:24.369919 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:24.370392 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:24.370406 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.371201 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.371204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:24.372099 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:24.372109 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.372673 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.372677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:24.372862 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.373470 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:24.373486 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:24.376219 (+ 2733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":111,"threads_started":1}
W20260806 00:49:24.377899 29039 heartbeater.cc:644] Failed to heartbeat to 127.15.147.190:45281 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.147.190:45281: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:24.384194 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:24.378435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:56395)
0806 00:49:24.378538 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:24.378554 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:24.378693 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:24.379026 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:24.379029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:24.379209 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:24.379304 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:24.379310 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.380162 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.380166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:24.380787 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:24.380797 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.381458 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.381460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:24.381529 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.382107 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:24.382118 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:24.384093 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 00:49:24.384482 28839 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:24.378526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:56395 (local address 127.15.147.190:45281)
0806 00:49:24.378661 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:24.378665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:24.378680 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:49:24.378820 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:24.378822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:24.378922 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:24.379010 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:24.379420 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.380061 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.380907 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.381198 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.381306 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.384298 (+ 2992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.130:56395: BlockingRecv error: recv got EOF from 127.15.147.130:56395 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:24.389739 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:24.384892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:48849)
0806 00:49:24.384984 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:24.384998 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:24.385088 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:24.385319 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:24.385322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:24.385472 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:24.385540 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:24.385544 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.386237 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.386242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:24.386750 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:24.386757 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.387192 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.387195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:24.387278 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.387786 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:24.387797 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:24.389600 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 00:49:24.389828 28839 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:24.384996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:48849 (local address 127.15.147.190:45281)
0806 00:49:24.385134 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:24.385139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:24.385154 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:49:24.385192 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:24.385194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:24.385225 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:24.385304 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:24.385646 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.386128 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.386864 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.387081 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.387181 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.389707 (+ 2526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.130:48849: BlockingRecv error: recv got EOF from 127.15.147.130:48849 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 00:49:24.390012 29039 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:49:24.503875 29045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:24.504228 29045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:24.504338 29045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:24.508739 29045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:24.508811 29045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:24.508910 29045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:49:24.514578 29045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:38769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/krb5kdc/oryx_127.15.147.131.keytab
--principal=oryx/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:37985
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=38611
--tserver_master_addrs=127.15.147.190:45281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:24.515766 29045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:24.516892 29045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:24.527438 29055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:24.527433 29052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:24.527436 29057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:24.527863 29045 server_base.cc:1034] running on GCE node
I20260806 00:49:24.528380 29045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:24.529089 29045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:24.530262 29045 hybrid_clock.cc:648] HybridClock initialized: now 1785977364530249 us; error 34 us; skew 500 ppm
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977364, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:24.533255 29045 init.cc:377] Logged in from keytab as oryx/127.15.147.131@KRBTEST.COM (short username oryx)
I20260806 00:49:24.534224 29045 webserver.cc:460] Webserver started at http://127.15.147.131:38611/ using document root <none> and password file <none>
I20260806 00:49:24.534822 29045 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:24.534905 29045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:24.539139 29045 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:49:24.541320 29065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:24.542539 29045 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:49:24.542667 29045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "0e1dffdcd7e946bea1e4ad0a1db6c69e"
format_stamp: "Formatted at 2026-08-06 00:49:23 on dist-test-slave-45dx"
server_key: "f7ce083f7bb9f0d885d5f6d96e1cb4ec"
server_key_iv: "5e0436b141955cdf947e708d96a1ffed"
server_key_version: "encryptionkey@0"
I20260806 00:49:24.543229 29045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:24.555939 29045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:24.559221 29045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:24.559451 29045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:24.560166 29045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:24.561113 29045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:24.561177 29045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:24.561259 29045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:24.561293 29045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:24.570751 29045 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:37985
I20260806 00:49:24.570863 29242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:37985 every 8 connection(s)
I20260806 00:49:24.571846 29045 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:49:24.574620 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 29045
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 00:49:24.584772 28839 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:24.573790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:54933 (local address 127.15.147.190:45281)
0806 00:49:24.573962 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:24.573965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:24.575340 (+ 1375us) server_negotiation.cc:407] Connection header received
0806 00:49:24.576628 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:24.576630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:24.576671 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:24.576740 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:24.578174 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.578777 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.580511 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.580744 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.580858 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.584631 (+ 3773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.131:54933: BlockingRecv error: recv got EOF from 127.15.147.131:54933 (error 108)
Metrics: {"server-negotiator.queue_time_us":60,"spinlock_wait_cycles":28416}
I20260806 00:49:24.585562 29247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:24.574269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.131:54933)
0806 00:49:24.575175 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:24.575230 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:24.576390 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:24.576969 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:24.576981 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:24.577393 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:24.577990 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:24.578011 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.578931 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.578934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:24.580373 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:24.580389 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.580907 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.580910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:24.581065 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.581713 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:24.581729 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:24.584513 (+ 2784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":284,"threads_started":1}
I20260806 00:49:24.585628 29249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:24.575719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45512 (local address 127.15.147.190:45281)
0806 00:49:24.575949 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:24.575953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:24.575965 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:49:24.576007 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:24.576009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:24.576049 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:24.576130 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:24.577149 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.577685 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.578725 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.579047 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.579125 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.580868 (+ 1743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:24.580875 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:24.580894 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:24.580932 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:24.583243 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:24.583721 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:24.583731 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:24.583737 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:24.583872 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:24.584190 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:24.584193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:24.584195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:24.584659 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:24.584896 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:24.585268 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 00:49:24.585505 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"spinlock_wait_cycles":1280,"thread_start_us":98,"threads_started":1}
W20260806 00:49:24.586086 29243 heartbeater.cc:644] Failed to heartbeat to 127.15.147.190:45281 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.147.190:45281: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:24.591918 29247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:24.586557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.131:36197)
0806 00:49:24.586694 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:24.586710 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:24.586796 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:24.587056 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:24.587060 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:24.587233 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:24.587354 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:24.587361 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.588266 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.588269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:24.589003 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:24.589014 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.589440 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.589443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:24.589511 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.590043 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:24.590055 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:24.591798 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 00:49:24.592062 29249 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:24.586662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:36197 (local address 127.15.147.190:45281)
0806 00:49:24.586761 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:24.586764 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:24.586777 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:49:24.586896 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:24.586898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:24.586935 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:24.587012 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:24.587473 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.588131 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.589116 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.589337 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.589426 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.591951 (+ 2525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.131:36197: BlockingRecv error: recv got EOF from 127.15.147.131:36197 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:24 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:24.597328 29247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:24.592634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.131:51997)
0806 00:49:24.592747 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:24.592759 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:24.592860 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:24.593087 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:24.593090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:24.593240 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:24.593310 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:24.593316 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.594017 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.594022 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:24.594532 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:24.594540 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.594977 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.594980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:24.595065 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.595498 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:24.595509 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:24.597228 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 00:49:24.597496 29249 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:24.592722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:51997 (local address 127.15.147.190:45281)
0806 00:49:24.592886 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:24.592893 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:24.592908 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:49:24.592947 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:24.592949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:24.592986 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:24.593072 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:24.593430 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.593913 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.594647 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:24.594870 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:24.594971 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:24.597344 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.131:51997: BlockingRecv error: recv got EOF from 127.15.147.131:51997 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
W20260806 00:49:24.597579 29243 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 00:49:25 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:25 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:25.182734 29250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:25.176937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:54719)
0806 00:49:25.177232 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:25.177250 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:25.177353 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:25.177591 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:25.177594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:25.177782 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:25.177937 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:25.177944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:25.178829 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:25.178832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:25.179415 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:25.179422 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:25.179937 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:25.179941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:25.180046 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:25.180672 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:25.180686 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:25.182619 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":77,"threads_started":1}
W20260806 00:49:25.183058 29251 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:25.177041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:54719 (local address 127.15.147.190:45281)
0806 00:49:25.177352 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:25.177356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:25.177369 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:49:25.177431 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:25.177433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:25.177474 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:25.177549 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:25.178044 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:25.178706 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:25.179532 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:25.179807 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:25.179890 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:25.182748 (+ 2858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.129:54719: BlockingRecv error: recv got EOF from 127.15.147.129:54719 (error 108)
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":80,"threads_started":1}
Aug 06 00:49:25 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:25 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:25.396752 29252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:25.390687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:49855)
0806 00:49:25.390998 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:25.391017 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:25.391175 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:25.391474 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:25.391478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:25.391698 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:25.391823 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:25.391830 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:25.392698 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:25.392705 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:25.393319 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:25.393328 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:25.393826 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:25.393830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:25.393927 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:25.394529 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:25.394549 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:25.396624 (+ 2075us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":83,"threads_started":1}
W20260806 00:49:25.396879 29251 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:25.390787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:49855 (local address 127.15.147.190:45281)
0806 00:49:25.390954 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:25.390959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:25.391098 (+ 139us) server_negotiation.cc:407] Connection header received
0806 00:49:25.391293 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:25.391296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:25.391352 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:25.391445 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:25.391954 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:25.392585 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:25.393457 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:25.393702 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:25.393820 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:25.396761 (+ 2941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.130:49855: BlockingRecv error: recv got EOF from 127.15.147.130:49855 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 00:49:25 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:25 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:25.603602 29253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:25.598347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.131:47331)
0806 00:49:25.598647 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:25.598662 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:25.598759 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:25.598964 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:25.598967 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:25.599150 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:25.599266 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:25.599272 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:25.599948 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:25.599950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:25.600592 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:25.600600 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:25.601008 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:25.601010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:25.601074 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:25.601643 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:25.601654 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:25.603489 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":119,"threads_started":1}
W20260806 00:49:25.603809 29251 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:25.598464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:47331 (local address 127.15.147.190:45281)
0806 00:49:25.598726 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:25.598730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:25.598744 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:49:25.598827 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:25.598829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:25.598873 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:25.598939 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:25.599357 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:25.599856 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:25.600681 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:25.600903 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:25.600978 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:25.603684 (+ 2706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.131:47331: BlockingRecv error: recv got EOF from 127.15.147.131:47331 (error 108)
Metrics: {"server-negotiator.queue_time_us":185}
Aug 06 00:49:26 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:26 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:26.189742 29254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:26.183646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:48895)
0806 00:49:26.183990 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:26.184004 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:26.184103 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:26.184459 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:26.184462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:26.184662 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:26.184864 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:26.184877 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:26.185681 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:26.185685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:26.186295 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:26.186305 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:26.186844 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:26.186848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:26.186947 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:26.187572 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:26.187585 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:26.189603 (+ 2018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":141,"threads_started":1}
W20260806 00:49:26.190018 29257 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:26.183781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:48895 (local address 127.15.147.190:45281)
0806 00:49:26.184054 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:26.184058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:26.184074 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:49:26.184209 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:26.184212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:26.184259 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:26.184370 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:26.185021 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:26.185564 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:26.186450 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:26.186696 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:26.186813 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:26.189747 (+ 2934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.129:48895: BlockingRecv error: recv got EOF from 127.15.147.129:48895 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":105,"threads_started":1}
Aug 06 00:49:26 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:26 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:26.408360 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:26.397559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:33353)
0806 00:49:26.397868 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:26.397884 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:26.397980 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:26.398213 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:26.398215 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:26.398408 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:26.398510 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:26.398516 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:26.399215 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:26.399217 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:26.399755 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:26.399762 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:26.400169 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:26.400172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:26.400235 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:26.406031 (+ 5796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:26.406046 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:26.408146 (+ 2100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":106,"threads_started":1}
W20260806 00:49:26.408504 29257 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:26.397754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:33353 (local address 127.15.147.190:45281)
0806 00:49:26.397920 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:26.397924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:26.397940 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:49:26.398054 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:26.398056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:26.398098 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:26.398172 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:26.398613 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:26.399121 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:26.399859 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:26.400068 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:26.400146 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:26.408377 (+ 8231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.130:33353: BlockingRecv error: recv got EOF from 127.15.147.130:33353 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 00:49:26 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:26 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:26.610016 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:26.604552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.131:44927)
0806 00:49:26.604952 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:26.604966 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:26.605074 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:26.605371 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:26.605373 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:26.605561 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:26.605661 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:26.605667 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:26.606399 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:26.606401 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:26.606967 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:26.606975 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:26.607378 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:26.607381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:26.607443 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:26.607959 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:26.607970 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:26.609886 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":125,"threads_started":1}
W20260806 00:49:26.610304 29257 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:26.604735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:44927 (local address 127.15.147.190:45281)
0806 00:49:26.604929 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:26.604933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:26.605041 (+ 108us) server_negotiation.cc:407] Connection header received
0806 00:49:26.605182 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:26.605188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:26.605250 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:26.605321 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:26.605760 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:26.606304 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:26.607068 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:26.607280 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:26.607354 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:26.610037 (+ 2683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.131:44927: BlockingRecv error: recv got EOF from 127.15.147.131:44927 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 00:49:27 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:27 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:27.196148 29260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:27.190592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:54861)
0806 00:49:27.190876 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:27.190890 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:27.190990 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:27.191251 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:27.191254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:27.191436 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:27.191537 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:27.191543 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:27.192280 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:27.192282 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:27.192902 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:27.192911 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:27.193336 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:27.193338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:27.193400 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:27.193990 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:27.194002 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:27.195956 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":102,"threads_started":1}
W20260806 00:49:27.196213 29261 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:27.190679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:54861 (local address 127.15.147.190:45281)
0806 00:49:27.190954 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:27.190957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:27.190971 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:49:27.191081 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:27.191083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:27.191129 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:27.191210 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:27.191644 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:27.192182 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:27.193007 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:27.193226 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:27.193314 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:27.196082 (+ 2768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.129:54861: BlockingRecv error: recv got EOF from 127.15.147.129:54861 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":116,"threads_started":1}
Aug 06 00:49:27 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:27 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:27.415194 29262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:27.409304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:44431)
0806 00:49:27.409596 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:27.409610 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:27.409717 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:27.409985 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:27.409988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:27.410177 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:27.410278 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:27.410283 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:27.411010 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:27.411012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:27.411548 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:27.411555 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:27.412132 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:27.412135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:27.412232 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:27.412884 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:27.412898 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:27.415008 (+ 2110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":103,"threads_started":1}
W20260806 00:49:27.415264 29261 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:27.409440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:44431 (local address 127.15.147.190:45281)
0806 00:49:27.409584 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:27.409588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:27.409684 (+ 96us) server_negotiation.cc:407] Connection header received
0806 00:49:27.409799 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:27.409801 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:27.409852 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:27.409932 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:27.410388 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:27.410906 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:27.411653 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:27.411998 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:27.412099 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:27.415151 (+ 3052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.130:44431: BlockingRecv error: recv got EOF from 127.15.147.130:44431 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 00:49:27 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:27 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:27.616499 29263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:27.610813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.131:48743)
0806 00:49:27.611147 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:27.611160 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:27.611267 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:27.611489 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:27.611491 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:27.611680 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:27.611777 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:27.611782 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:27.612623 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:27.612627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:27.613228 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:27.613238 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:27.613735 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:27.613738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:27.613828 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:27.614329 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:27.614341 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:27.616269 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":111,"threads_started":1}
W20260806 00:49:27.616597 29261 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:27.610917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:48743 (local address 127.15.147.190:45281)
0806 00:49:27.611135 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:27.611140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:27.611214 (+ 74us) server_negotiation.cc:407] Connection header received
0806 00:49:27.611342 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:27.611344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:27.611392 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:27.611450 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:27.611894 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:27.612484 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:27.613359 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:27.613607 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:27.613722 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:27.616437 (+ 2715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.131:48743: BlockingRecv error: recv got EOF from 127.15.147.131:48743 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Aug 06 00:49:28 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:28 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:28.202566 29264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:28.197019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:37905)
0806 00:49:28.197260 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:28.197274 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:28.197426 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:28.197796 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:28.197798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:28.197997 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:28.198095 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:28.198100 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:28.198812 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:28.198815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:28.199383 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:28.199391 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:28.199798 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:28.199800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:28.199866 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:28.200489 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:28.200501 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:28.202395 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":68,"threads_started":1}
W20260806 00:49:28.202641 29271 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:28.197343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:37905 (local address 127.15.147.190:45281)
0806 00:49:28.197593 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:28.197596 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:28.197609 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:49:28.197644 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:28.197646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:28.197685 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:28.197761 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:28.198198 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:28.198708 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:28.199485 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:28.199700 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:28.199774 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:28.202516 (+ 2742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.129:37905: BlockingRecv error: recv got EOF from 127.15.147.129:37905 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":91,"threads_started":1}
Aug 06 00:49:28 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:28 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:28.421568 29272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:28.416178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:36285)
0806 00:49:28.416524 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:28.416541 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:28.416658 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:28.416915 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:28.416917 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:28.417117 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:28.417213 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:28.417218 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:28.417952 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:28.417955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:28.418488 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:28.418495 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:28.418965 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:28.418967 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:28.419030 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:28.419506 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:28.419518 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:28.421457 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":132,"threads_started":1}
W20260806 00:49:28.421737 29271 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:28.416235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:36285 (local address 127.15.147.190:45281)
0806 00:49:28.416381 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:28.416385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:28.416636 (+ 251us) server_negotiation.cc:407] Connection header received
0806 00:49:28.416738 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:28.416740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:28.416789 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:28.416877 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:28.417315 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:28.417861 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:28.418606 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:28.418861 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:28.418941 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:28.421587 (+ 2646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.130:36285: BlockingRecv error: recv got EOF from 127.15.147.130:36285 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 00:49:28 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:28 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:28.623108 29273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:28.617403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.131:56677)
0806 00:49:28.617684 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:28.617697 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:28.617800 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:28.618150 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:28.618154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:28.618414 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:28.618559 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:28.618567 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:28.619611 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:28.619614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:28.620156 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:28.620176 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:28.620618 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:28.620621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:28.620682 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:28.621165 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:28.621176 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:28.622994 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":80,"threads_started":1}
W20260806 00:49:28.623344 29271 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:28.617507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:56677 (local address 127.15.147.190:45281)
0806 00:49:28.617691 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:28.617695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:28.617771 (+ 76us) server_negotiation.cc:407] Connection header received
0806 00:49:28.617932 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:28.617936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:28.617994 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:28.618084 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:28.618704 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:28.619509 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:28.620265 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:28.620524 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:28.620597 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:28.623150 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.131:56677: BlockingRecv error: recv got EOF from 127.15.147.131:56677 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 06 00:49:29 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:29 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:29.209232 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:29.203333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:36245)
0806 00:49:29.203630 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:29.203645 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:29.203732 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:29.204172 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:29.204175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:29.204472 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:29.204579 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:29.204585 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:29.205413 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:29.205416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:29.206001 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:29.206010 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:29.206433 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:29.206436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:29.206496 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:29.207016 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:29.207028 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:29.209057 (+ 2029us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":74,"threads_started":1}
W20260806 00:49:29.209299 29275 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:29.203417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:36245 (local address 127.15.147.190:45281)
0806 00:49:29.203690 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:29.203693 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:29.203706 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:49:29.203815 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:29.203817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:29.203859 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:29.204084 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:29.204722 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:29.205290 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:29.206104 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:29.206333 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:29.206408 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:29.209177 (+ 2769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.129:36245: BlockingRecv error: recv got EOF from 127.15.147.129:36245 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":85,"threads_started":1}
Aug 06 00:49:29 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:29 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:29.427939 29277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:29.422353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:57947)
0806 00:49:29.422718 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:29.422734 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:29.422881 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:29.423163 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:29.423166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:29.423416 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:29.423515 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:29.423522 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:29.424258 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:29.424263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:29.424909 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:29.424918 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:29.425386 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:29.425390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:29.425482 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:29.425999 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:29.426009 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:29.427801 (+ 1792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":139,"threads_started":1}
W20260806 00:49:29.428103 29275 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:29.422524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:57947 (local address 127.15.147.190:45281)
0806 00:49:29.422707 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:29.422712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:29.422822 (+ 110us) server_negotiation.cc:407] Connection header received
0806 00:49:29.423000 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:29.423009 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:29.423054 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:29.423140 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:29.423633 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:29.424148 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:29.425035 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:29.425272 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:29.425379 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:29.427946 (+ 2567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.130:57947: BlockingRecv error: recv got EOF from 127.15.147.130:57947 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 06 00:49:29 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:29 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:29.630055 29278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:29.624176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.131:53037)
0806 00:49:29.624530 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:29.624563 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:29.624710 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:29.625019 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:29.625022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:29.625275 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:29.625379 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:29.625385 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:29.626128 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:29.626131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:29.626736 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:29.626745 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:29.627170 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:29.627172 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:29.627232 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:29.627752 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:29.627767 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:29.629917 (+ 2150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":158,"threads_started":1}
W20260806 00:49:29.630396 29275 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:29.624244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:53037 (local address 127.15.147.190:45281)
0806 00:49:29.624402 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:29.624407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:29.624685 (+ 278us) server_negotiation.cc:407] Connection header received
0806 00:49:29.624818 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:29.624822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:29.624895 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:29.624978 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:29.625487 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:29.626017 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:29.626856 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:29.627071 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:29.627147 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:29.630099 (+ 2952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.131:53037: BlockingRecv error: recv got EOF from 127.15.147.131:53037 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 00:49:30 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:30 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:30.216197 29279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:30.210231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:40477)
0806 00:49:30.210558 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:30.210578 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:30.210720 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:30.211088 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:30.211092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:30.211329 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:30.211432 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:30.211439 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:30.212353 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:30.212358 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:30.212982 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:30.212990 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:30.213512 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:30.213516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:30.213612 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:30.214148 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:30.214163 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:30.216085 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":126,"threads_started":1}
W20260806 00:49:30.216449 29282 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:30.210402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:40477 (local address 127.15.147.190:45281)
0806 00:49:30.210694 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:30.210698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:30.210712 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:49:30.210868 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:30.210872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:30.210937 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:30.211059 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:30.211567 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:30.212185 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:30.213114 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:30.213383 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:30.213507 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:30.216228 (+ 2721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.129:40477: BlockingRecv error: recv got EOF from 127.15.147.129:40477 (error 108)
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":101,"threads_started":1}
Aug 06 00:49:30 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:30 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:30.434285 29283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:30.428743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:34941)
0806 00:49:30.429018 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:30.429031 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:30.429163 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:30.429481 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:30.429484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:30.429705 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:30.429802 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:30.429807 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:30.430583 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:30.430588 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:30.431157 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:30.431165 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:30.431663 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:30.431667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:30.431758 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:30.432286 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:30.432320 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:30.434168 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":82,"threads_started":1}
W20260806 00:49:30.434450 29282 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:30.428795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:34941 (local address 127.15.147.190:45281)
0806 00:49:30.428934 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:30.428941 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:30.429115 (+ 174us) server_negotiation.cc:407] Connection header received
0806 00:49:30.429279 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:30.429282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:30.429360 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:30.429449 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:30.429940 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:30.430471 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:30.431280 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:30.431539 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:30.431646 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:30.434312 (+ 2666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.130:34941: BlockingRecv error: recv got EOF from 127.15.147.130:34941 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 00:49:30 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:30 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:30.636271 29284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:30.630869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.131:43503)
0806 00:49:30.631166 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:30.631180 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:30.631278 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:30.631517 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:30.631520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:30.631711 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:30.631807 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:30.631812 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:30.632584 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:30.632587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:30.633123 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:30.633130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:30.633584 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:30.633586 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:30.633642 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:30.634111 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:30.634121 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:30.636098 (+ 1977us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":121,"threads_started":1}
W20260806 00:49:30.636339 29282 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:30.631001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:43503 (local address 127.15.147.190:45281)
0806 00:49:30.631160 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:30.631164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:30.631248 (+ 84us) server_negotiation.cc:407] Connection header received
0806 00:49:30.631359 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:30.631361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:30.631407 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:30.631476 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:30.631906 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:30.632481 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:30.633216 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:30.633486 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:30.633562 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:30.636214 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.131:43503: BlockingRecv error: recv got EOF from 127.15.147.131:43503 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 00:49:31 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:31 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:31.222306 29286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:31.217069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:40253)
0806 00:49:31.217445 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:31.217459 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:31.217567 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:31.217818 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:31.217820 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:31.218024 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:31.218138 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:31.218143 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:31.218834 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:31.218836 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:31.219346 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:31.219352 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:31.219734 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:31.219736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:31.219795 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:31.220383 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:31.220394 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:31.222193 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":91,"threads_started":1}
W20260806 00:49:31.222476 29287 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:31.217196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:40253 (local address 127.15.147.190:45281)
0806 00:49:31.217459 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:31.217463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:31.217534 (+ 71us) server_negotiation.cc:407] Connection header received
0806 00:49:31.217646 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:31.217649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:31.217694 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:31.217771 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:31.218236 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:31.218748 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:31.219436 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:31.219639 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:31.219708 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:31.222310 (+ 2602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.129:40253: BlockingRecv error: recv got EOF from 127.15.147.129:40253 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":63,"threads_started":1}
Aug 06 00:49:31 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:31 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:31.454366 29290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:31.435200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:44011)
0806 00:49:31.435540 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:31.435553 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:31.435653 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:31.435908 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:31.435910 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:31.436095 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:31.436190 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:31.436195 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:31.436939 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:31.436942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:31.437483 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:31.437491 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:31.438133 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:31.438136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:31.438227 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:31.452340 (+ 14113us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:31.452353 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:31.454242 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":126,"threads_started":1}
W20260806 00:49:31.454510 29287 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:31.435253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:44011 (local address 127.15.147.190:45281)
0806 00:49:31.435408 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:31.435412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:31.435623 (+ 211us) server_negotiation.cc:407] Connection header received
0806 00:49:31.435729 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:31.435732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:31.435775 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:31.435865 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:31.436288 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:31.436799 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:31.437630 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:31.437983 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:31.438129 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:31.454392 (+ 16263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.130:44011: BlockingRecv error: recv got EOF from 127.15.147.130:44011 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 00:49:31 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:31 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:31.642765 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:31.637087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.131:42089)
0806 00:49:31.637473 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:31.637487 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:31.637633 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:31.637924 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:31.637926 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:31.638159 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:31.638249 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:31.638255 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:31.639033 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:31.639037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:31.639596 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:31.639603 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:31.640106 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:31.640110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:31.640195 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:31.640722 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:31.640732 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:31.642603 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":86,"threads_started":1}
W20260806 00:49:31.643213 29287 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:31.637206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:42089 (local address 127.15.147.190:45281)
0806 00:49:31.637385 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:31.637390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:31.637569 (+ 179us) server_negotiation.cc:407] Connection header received
0806 00:49:31.637735 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:31.637739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:31.637796 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:31.637894 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:31.638385 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:31.638920 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:31.639751 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:31.639983 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:31.640100 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:31.643089 (+ 2989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.131:42089: BlockingRecv error: recv got EOF from 127.15.147.131:42089 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 06 00:49:32 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:32 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:32.228845 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:32.223007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:48937)
0806 00:49:32.223326 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:32.223379 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:32.223486 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:32.223743 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:32.223745 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:32.223942 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:32.224067 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:32.224072 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:32.225039 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:32.225041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:32.225614 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:32.225622 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:32.226083 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:32.226086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:32.226156 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:32.226640 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:32.226652 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:32.228689 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":91,"threads_started":1}
W20260806 00:49:32.228928 29294 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:32.223172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:48937 (local address 127.15.147.190:45281)
0806 00:49:32.223441 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:32.223445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:32.223459 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:49:32.223572 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:32.223574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:32.223619 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:32.223699 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:32.224199 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:32.224902 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:32.225721 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:32.225954 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:32.226038 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:32.228820 (+ 2782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.129:48937: BlockingRecv error: recv got EOF from 127.15.147.129:48937 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":101,"threads_started":1}
Aug 06 00:49:32 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:32 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:32.461030 29297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:32.455397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:55947)
0806 00:49:32.455694 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:32.455708 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:32.455808 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:32.456051 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:32.456053 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:32.456241 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:32.456378 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:32.456384 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:32.457269 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:32.457271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:32.457828 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:32.457835 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:32.458259 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:32.458261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:32.458322 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:32.458827 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:32.458838 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:32.460873 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":122,"threads_started":1}
W20260806 00:49:32.461162 29294 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:32.455524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:55947 (local address 127.15.147.190:45281)
0806 00:49:32.455669 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:32.455672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:32.455779 (+ 107us) server_negotiation.cc:407] Connection header received
0806 00:49:32.455885 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:32.455887 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:32.455932 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:32.456010 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:32.456475 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:32.457168 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:32.457934 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:32.458159 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:32.458232 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:32.461019 (+ 2787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.130:55947: BlockingRecv error: recv got EOF from 127.15.147.130:55947 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 00:49:32 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:32 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:32.649933 29298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:32.643762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.131:43961)
0806 00:49:32.644169 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:32.644186 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:32.644369 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:32.644702 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:32.644704 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:32.644987 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:32.645103 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:32.645110 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:32.645935 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:32.645938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:32.646513 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:32.646521 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:32.647047 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:32.647050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:32.647117 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:32.647743 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:32.647756 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:32.649800 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":138,"threads_started":1}
W20260806 00:49:32.650231 29294 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:32.643949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:43961 (local address 127.15.147.190:45281)
0806 00:49:32.644169 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:32.644175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:32.644272 (+ 97us) server_negotiation.cc:407] Connection header received
0806 00:49:32.644475 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:32.644479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:32.644562 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:32.644657 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:32.645194 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:32.645794 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:32.646616 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:32.646911 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:32.647033 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:32.649959 (+ 2926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.131:43961: BlockingRecv error: recv got EOF from 127.15.147.131:43961 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
Aug 06 00:49:33 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:33 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:33.235100 29299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:33.229737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:51943)
0806 00:49:33.230033 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:33.230047 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:33.230142 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:33.230386 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:33.230388 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:33.230569 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:33.230668 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:33.230673 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:33.231424 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:33.231426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:33.231973 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:33.231980 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:33.232473 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:33.232476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:33.232538 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:33.233129 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:33.233141 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:33.234958 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":83,"threads_started":1}
W20260806 00:49:33.235245 29301 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:33.229823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:51943 (local address 127.15.147.190:45281)
0806 00:49:33.230097 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:33.230101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:33.230114 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:49:33.230227 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:33.230230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:33.230274 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:33.230367 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:33.230784 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:33.231307 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:33.232068 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:33.232284 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:33.232440 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:33.235111 (+ 2671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.129:51943: BlockingRecv error: recv got EOF from 127.15.147.129:51943 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":74,"threads_started":1}
Aug 06 00:49:33 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:33 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:33.467917 29304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:33.461973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:49145)
0806 00:49:33.462348 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:33.462367 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:33.462511 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:33.462816 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:33.462819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:33.463060 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:33.463155 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:33.463161 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:33.463943 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:33.463948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:33.464531 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:33.464540 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:33.465005 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:33.465009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:33.465097 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:33.465638 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:33.465651 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:33.467707 (+ 2056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":101,"threads_started":1}
W20260806 00:49:33.467955 29301 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:33.462106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:49145 (local address 127.15.147.190:45281)
0806 00:49:33.462297 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:33.462301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:33.462446 (+ 145us) server_negotiation.cc:407] Connection header received
0806 00:49:33.462636 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:33.462640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:33.462696 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:33.462789 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:33.463279 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:33.463802 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:33.464653 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:33.464879 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:33.464988 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:33.467846 (+ 2858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.130:49145: BlockingRecv error: recv got EOF from 127.15.147.130:49145 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 06 00:49:33 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:33 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:33.656392 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:33.650645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.131:60259)
0806 00:49:33.650918 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:33.650931 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:33.651035 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:33.651259 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:33.651261 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:33.651458 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:33.651556 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:33.651561 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:33.652361 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:33.652368 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:33.653089 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:33.653099 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:33.653641 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:33.653645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:33.653728 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:33.654215 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:33.654227 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:33.656194 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":76,"threads_started":1}
W20260806 00:49:33.656484 29301 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:33.650743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:60259 (local address 127.15.147.190:45281)
0806 00:49:33.650926 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:33.650930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:33.650986 (+ 56us) server_negotiation.cc:407] Connection header received
0806 00:49:33.651114 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:33.651116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:33.651159 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:33.651219 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:33.651660 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:33.652186 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:33.653211 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:33.653521 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:33.653637 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:33.656363 (+ 2726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.131:60259: BlockingRecv error: recv got EOF from 127.15.147.131:60259 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 06 00:49:34 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:34 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:34.241868 29307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:34.236061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:35515)
0806 00:49:34.236409 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:34.236423 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:34.236568 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:34.236886 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:34.236891 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:34.237157 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:34.237291 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:34.237301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:34.238053 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:34.238056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:34.238632 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:34.238638 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:34.239071 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:34.239074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:34.239135 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:34.239700 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:34.239711 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:34.241678 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":112,"threads_started":1}
W20260806 00:49:34.241942 29308 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:34.236125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:35515 (local address 127.15.147.190:45281)
0806 00:49:34.236424 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:34.236430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:34.236512 (+ 82us) server_negotiation.cc:407] Connection header received
0806 00:49:34.236677 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:34.236683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:34.236741 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:34.236835 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:34.237422 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:34.237958 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:34.238730 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:34.238965 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:34.239047 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:34.241819 (+ 2772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.129:35515: BlockingRecv error: recv got EOF from 127.15.147.129:35515 (error 108)
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":83,"threads_started":1}
Aug 06 00:49:34 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:34 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:34.474889 29310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:34.468855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:59791)
0806 00:49:34.469208 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:34.469224 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:34.469376 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:34.469709 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:34.469712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:34.469998 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:34.470148 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:34.470159 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:34.471158 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:34.471160 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:34.471693 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:34.471700 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:34.472140 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:34.472143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:34.472205 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:34.472786 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:34.472807 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:34.474700 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":91,"threads_started":1}
W20260806 00:49:34.474921 29308 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:34.468981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:59791 (local address 127.15.147.190:45281)
0806 00:49:34.469161 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:34.469167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:34.469329 (+ 162us) server_negotiation.cc:407] Connection header received
0806 00:49:34.469504 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:34.469506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:34.469582 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:34.469665 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:34.470272 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:34.471054 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:34.471821 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:34.472033 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:34.472119 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:34.474824 (+ 2705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.130:59791: BlockingRecv error: recv got EOF from 127.15.147.130:59791 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 06 00:49:34 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:34 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:34.662985 29312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:34.657211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.131:56043)
0806 00:49:34.657529 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:34.657545 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:34.657656 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:34.657968 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:34.657970 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:34.658189 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:34.658283 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:34.658289 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:34.659140 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:34.659145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:34.659704 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:34.659712 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:34.660175 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:34.660179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:34.660264 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:34.660802 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:34.660814 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:34.662820 (+ 2006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":92,"threads_started":1}
W20260806 00:49:34.663022 29308 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:34.657351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:56043 (local address 127.15.147.190:45281)
0806 00:49:34.657516 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:34.657521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:34.657606 (+ 85us) server_negotiation.cc:407] Connection header received
0806 00:49:34.657787 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:34.657793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:34.657856 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:34.657943 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:34.658412 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:34.659026 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:34.659826 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:34.660054 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:34.660160 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:34.662921 (+ 2761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.131:56043: BlockingRecv error: recv got EOF from 127.15.147.131:56043 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 00:49:35 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:35 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:35.248497 29313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:35.242749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:43199)
0806 00:49:35.243028 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:35.243054 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:35.243145 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:35.243444 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:35.243447 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:35.243636 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:35.243738 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:35.243743 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:35.244578 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:35.244580 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:35.245123 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:35.245130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:35.245552 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:35.245554 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:35.245636 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:35.246242 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:35.246254 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:35.248365 (+ 2111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":102,"threads_started":1}
W20260806 00:49:35.248677 29316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:35.242825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:43199 (local address 127.15.147.190:45281)
0806 00:49:35.243088 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:35.243092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:35.243106 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:49:35.243230 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:35.243232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:35.243306 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:35.243392 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:35.243857 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:35.244474 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:35.245223 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:35.245453 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:35.245531 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:35.248492 (+ 2961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.129:43199: BlockingRecv error: recv got EOF from 127.15.147.129:43199 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":94,"threads_started":1}
Aug 06 00:49:35 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:35 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:35.482241 29319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:35.475873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:53611)
0806 00:49:35.476198 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:35.476213 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:35.476355 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:35.476615 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:35.476617 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:35.476850 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:35.476975 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:35.476981 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:35.478101 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:35.478104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:35.478929 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:35.478941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:35.479445 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:35.479449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:35.479522 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:35.480139 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:35.480151 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:35.482121 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":111,"threads_started":1}
W20260806 00:49:35.482437 29316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:35.475947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:53611 (local address 127.15.147.190:45281)
0806 00:49:35.476108 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:35.476111 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:35.476315 (+ 204us) server_negotiation.cc:407] Connection header received
0806 00:49:35.476442 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:35.476445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:35.476494 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:35.476567 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:35.477126 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:35.477973 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:35.479059 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:35.479327 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:35.479407 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:35.482288 (+ 2881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.130:53611: BlockingRecv error: recv got EOF from 127.15.147.130:53611 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 06 00:49:35 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:35 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:35.669698 29320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:35.663815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.131:58769)
0806 00:49:35.664162 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:35.664174 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:35.664326 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:35.664606 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:35.664609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:35.664822 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:35.664953 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:35.664959 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:35.665733 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:35.665738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:35.666298 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:35.666306 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:35.666808 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:35.666812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:35.666923 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:35.667461 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:35.667473 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:35.669517 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":115,"threads_started":1}
W20260806 00:49:35.669831 29316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:35.663896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:58769 (local address 127.15.147.190:45281)
0806 00:49:35.664101 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:35.664107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:35.664252 (+ 145us) server_negotiation.cc:407] Connection header received
0806 00:49:35.664434 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:35.664437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:35.664488 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:35.664579 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:35.665080 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:35.665618 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:35.666440 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:35.666692 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:35.666802 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:35.669672 (+ 2870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.131:58769: BlockingRecv error: recv got EOF from 127.15.147.131:58769 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 06 00:49:36 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:36 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:36.255018 29321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:36.249285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:43605)
0806 00:49:36.249573 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:36.249610 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:36.249718 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:36.250028 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:36.250031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:36.250255 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:36.250357 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:36.250363 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:36.251168 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:36.251172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:36.251746 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:36.251754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:36.252273 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:36.252277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:36.252454 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:36.252991 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:36.253003 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:36.254902 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":101,"threads_started":1}
W20260806 00:49:36.255198 29325 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:36.249378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:43605 (local address 127.15.147.190:45281)
0806 00:49:36.249699 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:36.249704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:36.249718 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:49:36.249850 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:36.249853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:36.249907 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:36.249998 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:36.250473 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:36.251053 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:36.251897 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:36.252149 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:36.252253 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:36.255055 (+ 2802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.129:43605: BlockingRecv error: recv got EOF from 127.15.147.129:43605 (error 108)
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":107,"threads_started":1}
Aug 06 00:49:36 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:36 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:36.493238 29326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:36.483372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:57323)
0806 00:49:36.483667 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:36.483703 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:36.483873 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:36.484179 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:36.484183 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:36.484514 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:36.484624 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:36.484630 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:36.485396 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:36.485400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:36.485976 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:36.485986 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:36.486413 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:36.486416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:36.486479 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:36.491028 (+ 4549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:36.491044 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:36.493105 (+ 2061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":95,"threads_started":1}
W20260806 00:49:36.493427 29325 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:36.483405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:57323 (local address 127.15.147.190:45281)
0806 00:49:36.483567 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:36.483573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:36.483816 (+ 243us) server_negotiation.cc:407] Connection header received
0806 00:49:36.483961 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:36.483964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:36.484021 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:36.484117 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:36.484751 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:36.485269 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:36.486088 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:36.486297 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:36.486381 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:36.493264 (+ 6883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.130:57323: BlockingRecv error: recv got EOF from 127.15.147.130:57323 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 00:49:36 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:36 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:36.676023 29327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:36.670502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.131:45849)
0806 00:49:36.670833 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:36.670848 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:36.670952 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:36.671177 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:36.671179 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:36.671394 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:36.671494 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:36.671499 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:36.672273 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:36.672276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:36.672874 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:36.672883 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:36.673310 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:36.673313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:36.673387 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:36.673963 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:36.673974 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:36.675906 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":90,"threads_started":1}
W20260806 00:49:36.676246 29325 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:36.670616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:45849 (local address 127.15.147.190:45281)
0806 00:49:36.670778 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:36.670789 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:36.670905 (+ 116us) server_negotiation.cc:407] Connection header received
0806 00:49:36.671019 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:36.671021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:36.671065 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:36.671135 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:36.671631 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:36.672165 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:36.672980 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:36.673209 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:36.673288 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:36.676078 (+ 2790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.131:45849: BlockingRecv error: recv got EOF from 127.15.147.131:45849 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 06 00:49:37 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:37 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:37.262210 29333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:37.255810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:42887)
0806 00:49:37.256169 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:37.256187 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:37.256369 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:37.256682 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:37.256686 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:37.256967 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:37.257131 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:37.257140 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:37.257960 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:37.257966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:37.258534 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:37.258542 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:37.259040 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:37.259045 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:37.259133 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:37.259771 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:37.259788 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:37.262081 (+ 2293us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":104,"threads_started":1}
W20260806 00:49:37.262367 29334 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:37.255872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:42887 (local address 127.15.147.190:45281)
0806 00:49:37.256196 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:37.256203 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:37.256275 (+ 72us) server_negotiation.cc:407] Connection header received
0806 00:49:37.256489 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:37.256494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:37.256554 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:37.256652 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:37.257269 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:37.257839 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:37.258667 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:37.258903 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:37.259011 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:37.262216 (+ 3205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.129:42887: BlockingRecv error: recv got EOF from 127.15.147.129:42887 (error 108)
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":89,"threads_started":1}
Aug 06 00:49:37 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:37 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:37.499519 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:37.494037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:55765)
0806 00:49:37.494321 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:37.494334 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:37.494456 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:37.494733 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:37.494736 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:37.494973 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:37.495104 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:37.495110 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:37.495863 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:37.495866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:37.496486 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:37.496495 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:37.496914 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:37.496917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:37.496980 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:37.497482 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:37.497494 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:37.499375 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":85,"threads_started":1}
W20260806 00:49:37.499701 29334 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:37.494181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:55765 (local address 127.15.147.190:45281)
0806 00:49:37.494338 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:37.494342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:37.494405 (+ 63us) server_negotiation.cc:407] Connection header received
0806 00:49:37.494538 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:37.494540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:37.494585 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:37.494651 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:37.495230 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:37.495752 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:37.496587 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:37.496806 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:37.496888 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:37.499545 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.130:55765: BlockingRecv error: recv got EOF from 127.15.147.130:55765 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 00:49:37 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:37 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:37.682287 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:37.676758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.131:46427)
0806 00:49:37.677088 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:37.677102 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:37.677203 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:37.677459 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:37.677461 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:37.677651 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:37.677749 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:37.677754 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:37.678558 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:37.678560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:37.679121 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:37.679129 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:37.679592 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:37.679595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:37.679670 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:37.680178 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:37.680189 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:37.682115 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":73,"threads_started":1}
W20260806 00:49:37.682348 29334 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:37.676840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:46427 (local address 127.15.147.190:45281)
0806 00:49:37.677053 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:37.677058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:37.677172 (+ 114us) server_negotiation.cc:407] Connection header received
0806 00:49:37.677285 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:37.677287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:37.677333 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:37.677428 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:37.677875 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:37.678460 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:37.679221 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:37.679492 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:37.679574 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:37.682235 (+ 2661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.131:46427: BlockingRecv error: recv got EOF from 127.15.147.131:46427 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Aug 06 00:49:38 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:38 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:38.270797 29339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:38.263186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:43205)
0806 00:49:38.263657 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:38.263674 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:38.263828 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:38.264173 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:38.264176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:38.264467 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:38.264619 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:38.264650 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:38.265929 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:38.265932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:38.267285 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:38.267296 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:38.268048 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:38.268055 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:38.268151 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:38.268724 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:38.268736 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:38.270670 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":80,"threads_started":1}
W20260806 00:49:38.270968 29342 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:38.263273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:43205 (local address 127.15.147.190:45281)
0806 00:49:38.263644 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:38.263650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:38.263782 (+ 132us) server_negotiation.cc:407] Connection header received
0806 00:49:38.263934 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:38.263951 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:38.264009 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:38.264121 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:38.264807 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:38.265784 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:38.267460 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:38.267879 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:38.268010 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:38.270829 (+ 2819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.129:43205: BlockingRecv error: recv got EOF from 127.15.147.129:43205 (error 108)
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":74,"threads_started":1}
Aug 06 00:49:38 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:38 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:38.506657 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:38.500440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:43121)
0806 00:49:38.500750 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:38.500767 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:38.500891 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:38.501224 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:38.501228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:38.501494 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:38.501608 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:38.501615 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:38.502443 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:38.502447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:38.503042 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:38.503050 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:38.503543 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:38.503546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:38.503653 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:38.504185 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:38.504195 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:38.506528 (+ 2333us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":110,"threads_started":1}
W20260806 00:49:38.506798 29342 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:38.500456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:43121 (local address 127.15.147.190:45281)
0806 00:49:38.500597 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:38.500601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:38.500856 (+ 255us) server_negotiation.cc:407] Connection header received
0806 00:49:38.501018 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:38.501023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:38.501091 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:38.501157 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:38.501729 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:38.502309 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:38.503179 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:38.503422 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:38.503555 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:38.506677 (+ 3122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.130:43121: BlockingRecv error: recv got EOF from 127.15.147.130:43121 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 00:49:38 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:38 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.131@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:38.688828 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:38.683040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.131:55175)
0806 00:49:38.683335 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:38.683349 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:38.683458 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:38.683706 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:38.683708 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:38.683924 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:38.684030 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:38.684036 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:38.684863 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:38.684866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:38.685417 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:38.685425 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:38.685845 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:38.685848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:38.685911 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:38.686437 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:38.686449 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:38.688669 (+ 2220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":112,"threads_started":1}
W20260806 00:49:38.689000 29342 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:38.683129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:55175 (local address 127.15.147.190:45281)
0806 00:49:38.683269 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:38.683273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:38.683421 (+ 148us) server_negotiation.cc:407] Connection header received
0806 00:49:38.683538 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:38.683541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:38.683588 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:38.683659 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:38.684143 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:38.684744 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:38.685517 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:38.685730 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:38.685805 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:38.688836 (+ 3031us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.131:55175: BlockingRecv error: recv got EOF from 127.15.147.131:55175 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 00:49:39 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:39 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.129@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:39.277009 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:39.271773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.129:54303)
0806 00:49:39.272083 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:39.272097 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:39.272184 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:39.272483 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:39.272485 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:39.272672 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:39.272769 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:39.272774 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:39.273493 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:39.273496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:39.274037 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:39.274044 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:39.274453 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:39.274455 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:39.274513 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:39.275012 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:39.275024 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:39.276808 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":118,"threads_started":1}
W20260806 00:49:39.277134 29346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:39.271838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:54303 (local address 127.15.147.190:45281)
0806 00:49:39.272120 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:39.272124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:39.272143 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:49:39.272276 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:39.272279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:39.272342 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:39.272429 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:39.272870 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:39.273401 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:39.274140 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:39.274354 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:39.274430 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:39.276986 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.129:54303: BlockingRecv error: recv got EOF from 127.15.147.129:54303 (error 108)
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":86,"threads_started":1}
Aug 06 00:49:39 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 00:49:39 dist-test-slave-45dx krb5kdc[27773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.147.130@KRBTEST.COM for oryx/127.15.147.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 00:49:39.512677 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:39.507496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:45281 (local address 127.15.147.130:33559)
0806 00:49:39.507773 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:39.507786 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:39.507896 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:39.508122 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:39.508124 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:39.508345 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:39.508454 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:39.508461 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:39.509169 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:39.509171 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:39.509673 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:39.509679 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:39.510062 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:39.510064 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:39.510123 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:39.510590 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:39.510601 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:39.512521 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.147.190:45281: Server oryx/127.15.147.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":111,"threads_started":1}
W20260806 00:49:39.512797 29346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:39.507572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:33559 (local address 127.15.147.190:45281)
0806 00:49:39.507742 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:39.507745 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:39.507837 (+ 92us) server_negotiation.cc:407] Connection header received
0806 00:49:39.507978 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:39.507980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:39.508024 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:39.508084 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:39.508560 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:39.509082 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:39.509761 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:39.509968 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:39.510040 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:39.512644 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.147.130:33559: BlockingRecv error: recv got EOF from 127.15.147.130:33559 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 00:49:39.584640 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 28615
I20260806 00:49:39.595698 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 28841
I20260806 00:49:39.605852 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 29045
I20260806 00:49:39.615386 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 28510
2026-08-06T00:49:39Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18309 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T00:49:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:49:39Z Disabled control of system clock
I20260806 00:49:39.652177 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:46157
--webserver_interface=127.15.147.190
--webserver_port=0
--builtin_ntp_servers=127.15.147.148:40721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:46157
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 00:49:39.762962 29351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:39.763242 29351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:39.763275 29351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:39.767189 29351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:49:39.767253 29351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:39.767269 29351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:39.767285 29351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:49:39.767300 29351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:49:39.771917 29351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:46157
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:46157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:39.772871 29351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:39.774118 29351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:39.782589 29363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:39.782640 29351 server_base.cc:1034] running on GCE node
W20260806 00:49:39.782594 29359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:39.782582 29357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:39.783398 29351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:39.784478 29351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:39.785681 29351 hybrid_clock.cc:648] HybridClock initialized: now 1785977379785653 us; error 36 us; skew 500 ppm
I20260806 00:49:39.787730 29351 webserver.cc:460] Webserver started at http://127.15.147.190:42463/ using document root <none> and password file <none>
I20260806 00:49:39.788462 29351 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:39.788532 29351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:39.788705 29351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:39.791015 29351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "294e06f2ad4841bdac6d678dc0b9d266"
format_stamp: "Formatted at 2026-08-06 00:49:39 on dist-test-slave-45dx"
server_key: "a1d93675e2fe43636824b3d2de89b0c2"
server_key_iv: "10974d7e36bd61913ba4ada86ea3e8f1"
server_key_version: "encryptionkey@0"
I20260806 00:49:39.791642 29351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "294e06f2ad4841bdac6d678dc0b9d266"
format_stamp: "Formatted at 2026-08-06 00:49:39 on dist-test-slave-45dx"
server_key: "a1d93675e2fe43636824b3d2de89b0c2"
server_key_iv: "10974d7e36bd61913ba4ada86ea3e8f1"
server_key_version: "encryptionkey@0"
I20260806 00:49:39.795764 29351 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:49:39.798661 29372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:39.800057 29351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:49:39.800179 29351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "294e06f2ad4841bdac6d678dc0b9d266"
format_stamp: "Formatted at 2026-08-06 00:49:39 on dist-test-slave-45dx"
server_key: "a1d93675e2fe43636824b3d2de89b0c2"
server_key_iv: "10974d7e36bd61913ba4ada86ea3e8f1"
server_key_version: "encryptionkey@0"
I20260806 00:49:39.800294 29351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:39.835886 29351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:39.836671 29351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:39.836822 29351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:39.843691 29351 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:46157
I20260806 00:49:39.843703 29443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:46157 every 8 connection(s)
I20260806 00:49:39.844645 29351 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:49:39.847689 29445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:39.848285 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 29351
I20260806 00:49:39.848424 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:49:39.848759 15950 external_mini_cluster.cc:1428] Setting key 8bf31c5fc8d46949420e99f8f4a39ae8
I20260806 00:49:39.853552 29445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266: Bootstrap starting.
I20260806 00:49:39.855579 29445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:39.856452 29445 log.cc:824] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:39.857647 29450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:39.850198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60496 (local address 127.15.147.190:46157)
0806 00:49:39.850788 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:39.850801 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:39.850833 (+ 32us) server_negotiation.cc:407] Connection header received
0806 00:49:39.851445 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:39.851460 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:39.851794 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:39.852105 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:39.852518 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:39.853422 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:39.854322 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:39.854763 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:39.854939 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:39.855758 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:39.855766 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:39.855781 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:39.856233 (+ 452us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:39.856319 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:39.856348 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:39.856453 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:39.856577 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 00:49:39.856819 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":146,"threads_started":1}
I20260806 00:49:39.858520 29445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266: No bootstrap required, opened a new log
I20260806 00:49:39.862001 29445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "294e06f2ad4841bdac6d678dc0b9d266" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46157 } }
I20260806 00:49:39.862269 29445 raft_consensus.cc:374] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:39.862365 29445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 294e06f2ad4841bdac6d678dc0b9d266, State: Initialized, Role: FOLLOWER
I20260806 00:49:39.862814 29445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "294e06f2ad4841bdac6d678dc0b9d266" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46157 } }
I20260806 00:49:39.862939 29445 raft_consensus.cc:388] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:49:39.863035 29445 raft_consensus.cc:482] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:49:39.863133 29445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:39.863965 29445 raft_consensus.cc:504] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "294e06f2ad4841bdac6d678dc0b9d266" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46157 } }
I20260806 00:49:39.864282 29445 leader_election.cc:302] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 294e06f2ad4841bdac6d678dc0b9d266; no voters:
I20260806 00:49:39.864639 29445 leader_election.cc:288] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:49:39.864753 29453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:39.865005 29453 raft_consensus.cc:686] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266 [term 1 LEADER]: Becoming Leader. State: Replica: 294e06f2ad4841bdac6d678dc0b9d266, State: Running, Role: LEADER
I20260806 00:49:39.865316 29453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "294e06f2ad4841bdac6d678dc0b9d266" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46157 } }
I20260806 00:49:39.865792 29445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:49:39.866910 29455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 294e06f2ad4841bdac6d678dc0b9d266. Latest consensus state: current_term: 1 leader_uuid: "294e06f2ad4841bdac6d678dc0b9d266" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "294e06f2ad4841bdac6d678dc0b9d266" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46157 } } }
I20260806 00:49:39.867039 29455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:39.867447 29469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:49:39.867741 29454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "294e06f2ad4841bdac6d678dc0b9d266" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "294e06f2ad4841bdac6d678dc0b9d266" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46157 } } }
I20260806 00:49:39.867844 29454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:39.870118 29469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:49:39.874926 29469 catalog_manager.cc:1347] Generated new cluster ID: f383152ea0a045338ceb5c913e5e60c0
I20260806 00:49:39.874987 29469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:49:39.881188 29469 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:49:39.882318 29469 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:49:39.893869 29469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 294e06f2ad4841bdac6d678dc0b9d266: Generated new TSK 0
I20260806 00:49:39.894539 29469 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:49:39.898424 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:46157
--builtin_ntp_servers=127.15.147.148:40721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 00:49:40.011499 29486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:40.011781 29486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:40.011826 29486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:40.015983 29486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:40.016038 29486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:40.016120 29486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:49:40.020903 29486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:46157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:40.021978 29486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:40.023367 29486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:40.033041 29491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:40.032999 29494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:40.033056 29492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:40.033341 29486 server_base.cc:1034] running on GCE node
I20260806 00:49:40.033825 29486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:40.034451 29486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:40.035648 29486 hybrid_clock.cc:648] HybridClock initialized: now 1785977380035605 us; error 53 us; skew 500 ppm
I20260806 00:49:40.037606 29486 webserver.cc:460] Webserver started at http://127.15.147.129:44975/ using document root <none> and password file <none>
I20260806 00:49:40.038196 29486 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:40.038262 29486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:40.038532 29486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:40.040763 29486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "ac92904d0ad2449593521799448f34c4"
format_stamp: "Formatted at 2026-08-06 00:49:40 on dist-test-slave-45dx"
server_key: "9eccb9e17450671396480e1ae912561d"
server_key_iv: "f1e3f20667e20968e444c2a67c0d2ca2"
server_key_version: "encryptionkey@0"
I20260806 00:49:40.041325 29486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "ac92904d0ad2449593521799448f34c4"
format_stamp: "Formatted at 2026-08-06 00:49:40 on dist-test-slave-45dx"
server_key: "9eccb9e17450671396480e1ae912561d"
server_key_iv: "f1e3f20667e20968e444c2a67c0d2ca2"
server_key_version: "encryptionkey@0"
I20260806 00:49:40.044899 29486 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:49:40.047530 29501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:40.048794 29486 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260806 00:49:40.048921 29486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "ac92904d0ad2449593521799448f34c4"
format_stamp: "Formatted at 2026-08-06 00:49:40 on dist-test-slave-45dx"
server_key: "9eccb9e17450671396480e1ae912561d"
server_key_iv: "f1e3f20667e20968e444c2a67c0d2ca2"
server_key_version: "encryptionkey@0"
I20260806 00:49:40.049023 29486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:40.057667 29486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:40.058267 29486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:40.058465 29486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:40.059027 29486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:40.059899 29486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:40.059945 29486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:40.060014 29486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:40.060045 29486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:40.070551 29486 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:35663
I20260806 00:49:40.070600 29667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:35663 every 8 connection(s)
I20260806 00:49:40.071939 29486 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:49:40.076421 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 29486
I20260806 00:49:40.076558 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:49:40.076954 15950 external_mini_cluster.cc:1428] Setting key b4e693cb5e7a4d39bc622430c3387c37
I20260806 00:49:40.079421 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:46157
--builtin_ntp_servers=127.15.147.148:40721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 00:49:40.085829 29450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:40.074315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:41973 (local address 127.15.147.190:46157)
0806 00:49:40.076208 (+ 1893us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:40.076213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:40.078194 (+ 1981us) server_negotiation.cc:407] Connection header received
0806 00:49:40.079992 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:40.079995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:40.080038 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:40.080137 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:40.081109 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:40.082003 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:40.083515 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:40.083900 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:40.084022 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:40.085211 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:40.085217 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:40.085221 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:40.085240 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:40.085269 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:40.085281 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:40.085365 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:40.085525 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 00:49:40.085599 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1786}
I20260806 00:49:40.086656 29672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:40.075055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:46157 (local address 127.15.147.129:41973)
0806 00:49:40.078014 (+ 2959us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:40.078072 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:40.079768 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:40.080361 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:40.080370 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:40.080465 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:40.080964 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:40.080978 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:40.082210 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:40.082216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:40.083390 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:40.083400 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:40.084049 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:40.084052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:40.084192 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:40.084935 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:49:40.084937 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:40.084977 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:49:40.085043 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:40.085376 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:40.085383 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:40.085388 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 00:49:40.085472 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 00:49:40.085703 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2462,"thread_start_us":891,"threads_started":1}
I20260806 00:49:40.088002 29670 heartbeater.cc:347] Connected to a master server at 127.15.147.190:46157
I20260806 00:49:40.088263 29670 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:40.088850 29670 heartbeater.cc:511] Master 127.15.147.190:46157 requested a full tablet report, sending...
I20260806 00:49:40.090672 29393 ts_manager.cc:194] Registered new tserver with Master: ac92904d0ad2449593521799448f34c4 (127.15.147.129:35663)
I20260806 00:49:40.091815 29393 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.147.129:41973
W20260806 00:49:40.198199 29673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:40.198500 29673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:40.198541 29673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:40.202783 29673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:40.202864 29673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:40.202960 29673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:49:40.207821 29673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:46157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:40.208971 29673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:40.210407 29673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:40.220498 29683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:40.220459 29679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:40.220535 29673 server_base.cc:1034] running on GCE node
W20260806 00:49:40.220490 29681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:40.221176 29673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:40.221817 29673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:40.222998 29673 hybrid_clock.cc:648] HybridClock initialized: now 1785977380222966 us; error 30 us; skew 500 ppm
I20260806 00:49:40.224897 29673 webserver.cc:460] Webserver started at http://127.15.147.130:45957/ using document root <none> and password file <none>
I20260806 00:49:40.225462 29673 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:40.225523 29673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:40.225775 29673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:40.227767 29673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "5b3e111d1f154b2ca258433c0763cd7a"
format_stamp: "Formatted at 2026-08-06 00:49:40 on dist-test-slave-45dx"
server_key: "99fb952a5c4352440921b35ae12cb06e"
server_key_iv: "5ac0815d5ff8c5439e84a666c3d57394"
server_key_version: "encryptionkey@0"
I20260806 00:49:40.228399 29673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "5b3e111d1f154b2ca258433c0763cd7a"
format_stamp: "Formatted at 2026-08-06 00:49:40 on dist-test-slave-45dx"
server_key: "99fb952a5c4352440921b35ae12cb06e"
server_key_iv: "5ac0815d5ff8c5439e84a666c3d57394"
server_key_version: "encryptionkey@0"
I20260806 00:49:40.231937 29673 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:49:40.234442 29690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:40.235597 29673 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:49:40.235729 29673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "5b3e111d1f154b2ca258433c0763cd7a"
format_stamp: "Formatted at 2026-08-06 00:49:40 on dist-test-slave-45dx"
server_key: "99fb952a5c4352440921b35ae12cb06e"
server_key_iv: "5ac0815d5ff8c5439e84a666c3d57394"
server_key_version: "encryptionkey@0"
I20260806 00:49:40.235839 29673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:40.250554 29673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:40.251187 29673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:40.251375 29673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:40.252009 29673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:40.252938 29673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:40.253017 29673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:40.253108 29673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:40.253159 29673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:40.262216 29673 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:43143
I20260806 00:49:40.262327 29867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:43143 every 8 connection(s)
I20260806 00:49:40.263170 29673 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:49:40.267545 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 29673
I20260806 00:49:40.267661 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:49:40.268028 15950 external_mini_cluster.cc:1428] Setting key b3d1bf007669786e230b9970cb069a44
I20260806 00:49:40.270327 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:46157
--builtin_ntp_servers=127.15.147.148:40721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 00:49:40.274490 29450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:40.265419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:50911 (local address 127.15.147.190:46157)
0806 00:49:40.265584 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:40.265589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:40.267072 (+ 1483us) server_negotiation.cc:407] Connection header received
0806 00:49:40.269023 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:40.269026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:40.269078 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:40.269166 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:40.270148 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:40.271013 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:40.272377 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:40.272595 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:40.272681 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:40.273948 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:40.273954 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:40.273959 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:40.273976 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:40.274004 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:40.274016 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:40.274101 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:40.274219 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 00:49:40.274290 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 00:49:40.275244 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:40.265993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:46157 (local address 127.15.147.130:50911)
0806 00:49:40.266906 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:40.266959 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:40.268800 (+ 1841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:40.269315 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:40.269323 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:40.269428 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:40.269978 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:40.269996 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:40.271140 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:40.271146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:40.272203 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:40.272217 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:40.272754 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:40.272760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:40.272981 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:40.273624 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:49:40.273626 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:40.273678 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:49:40.273758 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:40.274098 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:40.274102 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:40.274105 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 00:49:40.274187 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 00:49:40.274404 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":155,"threads_started":1}
I20260806 00:49:40.276369 29868 heartbeater.cc:347] Connected to a master server at 127.15.147.190:46157
I20260806 00:49:40.276636 29868 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:40.277303 29868 heartbeater.cc:511] Master 127.15.147.190:46157 requested a full tablet report, sending...
I20260806 00:49:40.278474 29393 ts_manager.cc:194] Registered new tserver with Master: 5b3e111d1f154b2ca258433c0763cd7a (127.15.147.130:43143)
I20260806 00:49:40.279047 29393 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.147.130:50911
W20260806 00:49:40.398535 29872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:40.398784 29872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:40.398855 29872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:40.402864 29872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:40.402923 29872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:40.402976 29872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:49:40.407917 29872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:46157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:40.409027 29872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:40.410392 29872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:40.420223 29880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:40.420284 29872 server_base.cc:1034] running on GCE node
W20260806 00:49:40.420205 29878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:40.420205 29882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:40.420939 29872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:40.421464 29872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:40.422641 29872 hybrid_clock.cc:648] HybridClock initialized: now 1785977380422610 us; error 50 us; skew 500 ppm
I20260806 00:49:40.424458 29872 webserver.cc:460] Webserver started at http://127.15.147.131:35797/ using document root <none> and password file <none>
I20260806 00:49:40.425016 29872 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:40.425109 29872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:40.425302 29872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:40.427248 29872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "87486cfe4a964fb49cd10ecc6bf31efa"
format_stamp: "Formatted at 2026-08-06 00:49:40 on dist-test-slave-45dx"
server_key: "daeeb3a6186b1da57a2b058706055070"
server_key_iv: "8319b405560e8408b7940c837a00eee8"
server_key_version: "encryptionkey@0"
I20260806 00:49:40.427824 29872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "87486cfe4a964fb49cd10ecc6bf31efa"
format_stamp: "Formatted at 2026-08-06 00:49:40 on dist-test-slave-45dx"
server_key: "daeeb3a6186b1da57a2b058706055070"
server_key_iv: "8319b405560e8408b7940c837a00eee8"
server_key_version: "encryptionkey@0"
I20260806 00:49:40.431593 29872 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.005s
I20260806 00:49:40.434020 29891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:40.435133 29872 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:49:40.435272 29872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "87486cfe4a964fb49cd10ecc6bf31efa"
format_stamp: "Formatted at 2026-08-06 00:49:40 on dist-test-slave-45dx"
server_key: "daeeb3a6186b1da57a2b058706055070"
server_key_iv: "8319b405560e8408b7940c837a00eee8"
server_key_version: "encryptionkey@0"
I20260806 00:49:40.435386 29872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:40.462531 29872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:40.463188 29872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:40.463375 29872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:40.463924 29872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:40.464874 29872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:40.464941 29872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:40.464999 29872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:40.465044 29872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:40.474059 29872 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:44805
I20260806 00:49:40.474088 30063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:44805 every 8 connection(s)
I20260806 00:49:40.474984 29872 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:49:40.478482 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 29872
I20260806 00:49:40.478598 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:49:40.478909 15950 external_mini_cluster.cc:1428] Setting key f0c4998c3241378f50012fad2c2f7a5a
I20260806 00:49:40.485003 29450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:40.476897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:58223 (local address 127.15.147.190:46157)
0806 00:49:40.477032 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:40.477035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:40.478059 (+ 1024us) server_negotiation.cc:407] Connection header received
0806 00:49:40.480004 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:40.480009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:40.480076 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:40.480148 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:40.481336 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:40.481926 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:40.483070 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:40.483281 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:40.483359 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:40.484562 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:40.484565 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:40.484567 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:40.484579 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:40.484600 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:40.484608 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:40.484679 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:40.484838 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 00:49:40.484890 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:49:40.485828 30068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:40.477222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:46157 (local address 127.15.147.131:58223)
0806 00:49:40.477905 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:40.477947 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:40.479723 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:40.480417 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:40.480424 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:40.480551 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:40.481154 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:40.481175 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:40.482051 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:40.482054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:40.482964 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:40.482974 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:40.483416 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:40.483418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:40.483562 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:40.484229 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:49:40.484231 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:40.484270 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:49:40.484389 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:40.484694 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:40.484702 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:40.484706 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 00:49:40.484803 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 00:49:40.485025 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":122,"threads_started":1}
I20260806 00:49:40.486931 30065 heartbeater.cc:347] Connected to a master server at 127.15.147.190:46157
I20260806 00:49:40.487138 30065 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:40.487581 30065 heartbeater.cc:511] Master 127.15.147.190:46157 requested a full tablet report, sending...
I20260806 00:49:40.488636 29393 ts_manager.cc:194] Registered new tserver with Master: 87486cfe4a964fb49cd10ecc6bf31efa (127.15.147.131:44805)
I20260806 00:49:40.489226 29393 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.147.131:58223
I20260806 00:49:40.493036 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 00:49:40.495915 29450 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:40.494956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60510 (local address 127.15.147.190:46157)
0806 00:49:40.495202 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:40.495208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:40.495223 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:49:40.495377 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:40.495380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:40.495421 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:40.495515 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:40.495776 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60510: BlockingRecv error: recv got EOF from 127.0.0.1:60510 (error 108)
Metrics: {"server-negotiator.queue_time_us":167}
I20260806 00:49:40.496178 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 29486
I20260806 00:49:40.519702 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 29673
I20260806 00:49:40.533291 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 29872
I20260806 00:49:40.549176 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 29351
2026-08-06T00:49:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (932 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T00:49:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:49:40Z Disabled control of system clock
I20260806 00:49:40.584591 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:46687
--webserver_interface=127.15.147.190
--webserver_port=0
--builtin_ntp_servers=127.15.147.148:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:46687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 00:49:40.694558 30078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:40.694804 30078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:40.694839 30078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:40.698681 30078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:49:40.698748 30078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:40.698764 30078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:40.698781 30078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:49:40.698796 30078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:49:40.703490 30078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:46687
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:46687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:40.704516 30078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:40.705801 30078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:49:40.714388 30078 server_base.cc:1034] running on GCE node
W20260806 00:49:40.714332 30088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:40.714408 30085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:40.714329 30086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:40.715181 30078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:40.716253 30078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:40.717541 30078 hybrid_clock.cc:648] HybridClock initialized: now 1785977380717512 us; error 38 us; skew 500 ppm
I20260806 00:49:40.719419 30078 webserver.cc:460] Webserver started at http://127.15.147.190:32939/ using document root <none> and password file <none>
I20260806 00:49:40.720038 30078 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:40.720130 30078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:40.720424 30078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:40.722594 30078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "35ea9e6bfb3d471ea79f75809a82be44"
format_stamp: "Formatted at 2026-08-06 00:49:40 on dist-test-slave-45dx"
server_key: "817dc05fcf5acfcff9dced4831e691f7"
server_key_iv: "3672d7838581e3f41f54fb899a39be1e"
server_key_version: "encryptionkey@0"
I20260806 00:49:40.723224 30078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "35ea9e6bfb3d471ea79f75809a82be44"
format_stamp: "Formatted at 2026-08-06 00:49:40 on dist-test-slave-45dx"
server_key: "817dc05fcf5acfcff9dced4831e691f7"
server_key_iv: "3672d7838581e3f41f54fb899a39be1e"
server_key_version: "encryptionkey@0"
I20260806 00:49:40.727057 30078 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260806 00:49:40.729553 30097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:40.730799 30078 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 00:49:40.730957 30078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "35ea9e6bfb3d471ea79f75809a82be44"
format_stamp: "Formatted at 2026-08-06 00:49:40 on dist-test-slave-45dx"
server_key: "817dc05fcf5acfcff9dced4831e691f7"
server_key_iv: "3672d7838581e3f41f54fb899a39be1e"
server_key_version: "encryptionkey@0"
I20260806 00:49:40.731072 30078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:40.735613 30078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:40.736210 30078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:40.736420 30078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:40.743285 30078 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:46687
I20260806 00:49:40.743299 30166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:46687 every 8 connection(s)
I20260806 00:49:40.744135 30078 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:49:40.747238 30167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:40.750411 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 30078
I20260806 00:49:40.750514 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:49:40.750834 15950 external_mini_cluster.cc:1428] Setting key ab57ea75e570e5e5d3f6c7621bccbbdd
I20260806 00:49:40.753165 30167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44: Bootstrap starting.
I20260806 00:49:40.755848 30167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:40.756714 30167 log.cc:824] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:40.756891 30171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:40.752273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52598 (local address 127.15.147.190:46687)
0806 00:49:40.752989 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:40.753008 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:40.753050 (+ 42us) server_negotiation.cc:407] Connection header received
0806 00:49:40.753831 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:40.753854 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:40.754268 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:40.754652 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:40.755467 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:40.755473 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:40.755487 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:40.755499 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:40.755562 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:40.755585 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:40.755700 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:40.755783 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 00:49:40.756033 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":117,"threads_started":1}
I20260806 00:49:40.758823 30167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44: No bootstrap required, opened a new log
I20260806 00:49:40.762301 30167 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ea9e6bfb3d471ea79f75809a82be44" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46687 } }
I20260806 00:49:40.762555 30167 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:40.762626 30167 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35ea9e6bfb3d471ea79f75809a82be44, State: Initialized, Role: FOLLOWER
I20260806 00:49:40.763069 30167 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ea9e6bfb3d471ea79f75809a82be44" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46687 } }
I20260806 00:49:40.763193 30167 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:49:40.763232 30167 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:49:40.763302 30167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:40.764097 30167 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ea9e6bfb3d471ea79f75809a82be44" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46687 } }
I20260806 00:49:40.764433 30167 leader_election.cc:302] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35ea9e6bfb3d471ea79f75809a82be44; no voters:
I20260806 00:49:40.764678 30167 leader_election.cc:288] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:49:40.764828 30174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:40.765105 30174 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44 [term 1 LEADER]: Becoming Leader. State: Replica: 35ea9e6bfb3d471ea79f75809a82be44, State: Running, Role: LEADER
I20260806 00:49:40.765401 30174 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ea9e6bfb3d471ea79f75809a82be44" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46687 } }
I20260806 00:49:40.765908 30167 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:49:40.767021 30177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35ea9e6bfb3d471ea79f75809a82be44. Latest consensus state: current_term: 1 leader_uuid: "35ea9e6bfb3d471ea79f75809a82be44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ea9e6bfb3d471ea79f75809a82be44" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46687 } } }
I20260806 00:49:40.767154 30177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:40.767009 30175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35ea9e6bfb3d471ea79f75809a82be44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ea9e6bfb3d471ea79f75809a82be44" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46687 } } }
I20260806 00:49:40.767406 30175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:40.767550 30190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:49:40.770740 30190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:49:40.776181 30190 catalog_manager.cc:1347] Generated new cluster ID: 3a53bcd91bee43e4bd7e71665783b7b7
I20260806 00:49:40.776264 30190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:49:40.795068 30190 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:49:40.795385 30190 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:49:40.801272 30190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35ea9e6bfb3d471ea79f75809a82be44: Generated new TSK 0
I20260806 00:49:40.801986 30190 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:49:40.811757 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:46687
--builtin_ntp_servers=127.15.147.148:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 00:49:40.926397 30206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:40.926656 30206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:40.926694 30206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:40.930703 30206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:40.930768 30206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:40.930822 30206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:49:40.935868 30206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:46687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:40.937032 30206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:40.938481 30206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:40.948165 30219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:40.948375 30206 server_base.cc:1034] running on GCE node
W20260806 00:49:40.948177 30213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:40.948203 30217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:40.949013 30206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:40.949602 30206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:40.950765 30206 hybrid_clock.cc:648] HybridClock initialized: now 1785977380950736 us; error 39 us; skew 500 ppm
I20260806 00:49:40.952685 30206 webserver.cc:460] Webserver started at http://127.15.147.129:33655/ using document root <none> and password file <none>
I20260806 00:49:40.953223 30206 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:40.953284 30206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:40.953457 30206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:40.955444 30206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "e2fd68fc28d74a1f933810e15d6db810"
format_stamp: "Formatted at 2026-08-06 00:49:40 on dist-test-slave-45dx"
server_key: "acf13ff975a749d2ea5643c1de403785"
server_key_iv: "25198f72e6dbb6c5dac8d9cfc30f8242"
server_key_version: "encryptionkey@0"
I20260806 00:49:40.955964 30206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "e2fd68fc28d74a1f933810e15d6db810"
format_stamp: "Formatted at 2026-08-06 00:49:40 on dist-test-slave-45dx"
server_key: "acf13ff975a749d2ea5643c1de403785"
server_key_iv: "25198f72e6dbb6c5dac8d9cfc30f8242"
server_key_version: "encryptionkey@0"
I20260806 00:49:40.959693 30206 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:49:40.962464 30228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:40.963780 30206 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 00:49:40.963914 30206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "e2fd68fc28d74a1f933810e15d6db810"
format_stamp: "Formatted at 2026-08-06 00:49:40 on dist-test-slave-45dx"
server_key: "acf13ff975a749d2ea5643c1de403785"
server_key_iv: "25198f72e6dbb6c5dac8d9cfc30f8242"
server_key_version: "encryptionkey@0"
I20260806 00:49:40.964049 30206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:40.969290 30206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:40.969957 30206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:40.970152 30206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:40.972225 30206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:40.973225 30206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:40.973271 30206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:40.973341 30206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:40.973371 30206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:40.982558 30206 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:37575
I20260806 00:49:40.982599 30392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:37575 every 8 connection(s)
I20260806 00:49:40.983446 30206 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:49:40.989338 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 30206
I20260806 00:49:40.989477 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:49:40.989774 30171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:40.985276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:52203 (local address 127.15.147.190:46687)
0806 00:49:40.985441 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:40.985446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:40.986523 (+ 1077us) server_negotiation.cc:407] Connection header received
0806 00:49:40.987878 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:40.987882 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:40.987930 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:40.988042 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:40.989175 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:40.989178 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:40.989181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:40.989193 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:40.989222 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:40.989230 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:40.989329 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:40.989589 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 00:49:40.989639 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:49:40.989790 15950 external_mini_cluster.cc:1428] Setting key 86db15d35f8d63f8c07c69ebf46a1daf
I20260806 00:49:40.990737 30396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:40.985721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:46687 (local address 127.15.147.129:52203)
0806 00:49:40.986372 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:40.986414 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:40.987645 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:40.988213 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:40.988219 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:40.988365 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:40.988916 (+ 551us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:49:40.988917 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:40.988956 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:49:40.989022 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:40.989313 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:40.989322 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:40.989328 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 00:49:40.989621 (+ 293us) client_negotiation.cc:240] Negotiation successful
0806 00:49:40.989856 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":105,"threads_started":1}
I20260806 00:49:40.991945 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:46687
--builtin_ntp_servers=127.15.147.148:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 00:49:40.991984 30394 heartbeater.cc:347] Connected to a master server at 127.15.147.190:46687
I20260806 00:49:40.992244 30394 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:40.992674 30394 heartbeater.cc:511] Master 127.15.147.190:46687 requested a full tablet report, sending...
I20260806 00:49:40.994233 30118 ts_manager.cc:194] Registered new tserver with Master: e2fd68fc28d74a1f933810e15d6db810 (127.15.147.129:37575)
W20260806 00:49:41.125789 30397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:41.126216 30397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:41.126307 30397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:41.133718 30397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:41.133785 30397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:41.133845 30397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:49:41.138985 30397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:46687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:41.140092 30397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:41.141594 30397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:41.151321 30409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:41.151309 30405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:41.151299 30404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:41.151635 30397 server_base.cc:1034] running on GCE node
I20260806 00:49:41.152072 30397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:41.152716 30397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:41.153898 30397 hybrid_clock.cc:648] HybridClock initialized: now 1785977381153865 us; error 42 us; skew 500 ppm
I20260806 00:49:41.155716 30397 webserver.cc:460] Webserver started at http://127.15.147.130:37705/ using document root <none> and password file <none>
I20260806 00:49:41.156261 30397 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:41.156391 30397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:41.156620 30397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:41.158582 30397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "de0a3c0275b04ebf8871a18903b5cb78"
format_stamp: "Formatted at 2026-08-06 00:49:41 on dist-test-slave-45dx"
server_key: "e62fd90d22d4195e64ce774a7c013ba0"
server_key_iv: "79c5837e5fae181051df50de198ec844"
server_key_version: "encryptionkey@0"
I20260806 00:49:41.159138 30397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "de0a3c0275b04ebf8871a18903b5cb78"
format_stamp: "Formatted at 2026-08-06 00:49:41 on dist-test-slave-45dx"
server_key: "e62fd90d22d4195e64ce774a7c013ba0"
server_key_iv: "79c5837e5fae181051df50de198ec844"
server_key_version: "encryptionkey@0"
I20260806 00:49:41.162737 30397 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 00:49:41.165114 30415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:41.166293 30397 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:49:41.166425 30397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "de0a3c0275b04ebf8871a18903b5cb78"
format_stamp: "Formatted at 2026-08-06 00:49:41 on dist-test-slave-45dx"
server_key: "e62fd90d22d4195e64ce774a7c013ba0"
server_key_iv: "79c5837e5fae181051df50de198ec844"
server_key_version: "encryptionkey@0"
I20260806 00:49:41.166534 30397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:41.171247 30397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:41.171828 30397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:41.172011 30397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:41.172647 30397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:41.173511 30397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:41.173616 30397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:41.173676 30397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:41.173724 30397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:41.182780 30397 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:38721
I20260806 00:49:41.182842 30587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:38721 every 8 connection(s)
I20260806 00:49:41.183701 30397 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:49:41.189554 30171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:41.185520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:60135 (local address 127.15.147.190:46687)
0806 00:49:41.185694 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:41.185699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:41.186713 (+ 1014us) server_negotiation.cc:407] Connection header received
0806 00:49:41.187971 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:41.187973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:41.188011 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:41.188085 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:41.189176 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:41.189179 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:41.189181 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:41.189193 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:41.189214 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:41.189222 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:41.189294 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:41.189413 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 00:49:41.189450 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 00:49:41.189955 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 30397
I20260806 00:49:41.190075 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:49:41.190377 15950 external_mini_cluster.cc:1428] Setting key cc05f32708fe33744ee45d60562b118a
I20260806 00:49:41.190469 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:41.185922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:46687 (local address 127.15.147.130:60135)
0806 00:49:41.186558 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:41.186599 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:41.187781 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:41.188240 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:41.188246 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:41.188387 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:41.188934 (+ 547us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:49:41.188936 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:41.188976 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:49:41.189041 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:41.189296 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:41.189302 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:41.189306 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 00:49:41.189429 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 00:49:41.189632 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":106,"threads_started":1}
I20260806 00:49:41.191608 30588 heartbeater.cc:347] Connected to a master server at 127.15.147.190:46687
I20260806 00:49:41.191838 30588 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:41.192253 30588 heartbeater.cc:511] Master 127.15.147.190:46687 requested a full tablet report, sending...
I20260806 00:49:41.192617 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:46687
--builtin_ntp_servers=127.15.147.148:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 00:49:41.193331 30117 ts_manager.cc:194] Registered new tserver with Master: de0a3c0275b04ebf8871a18903b5cb78 (127.15.147.130:38721)
W20260806 00:49:41.307628 30592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:41.307874 30592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:41.307912 30592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:41.311857 30592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:41.311915 30592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:41.311973 30592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:49:41.316807 30592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:46687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:41.317796 30592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:41.319129 30592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:41.328742 30601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:41.328750 30604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:41.328930 30592 server_base.cc:1034] running on GCE node
W20260806 00:49:41.328750 30600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:41.329409 30592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:41.329958 30592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:41.331110 30592 hybrid_clock.cc:648] HybridClock initialized: now 1785977381331081 us; error 39 us; skew 500 ppm
I20260806 00:49:41.332867 30592 webserver.cc:460] Webserver started at http://127.15.147.131:40789/ using document root <none> and password file <none>
I20260806 00:49:41.333441 30592 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:41.333503 30592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:41.333752 30592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:41.335963 30592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "9727470a859c47588c311b0f3a197546"
format_stamp: "Formatted at 2026-08-06 00:49:41 on dist-test-slave-45dx"
server_key: "0441ae91678b870d69c54081db79e157"
server_key_iv: "4e9ef44717f7b9e57bb1e1b39a1e97cb"
server_key_version: "encryptionkey@0"
I20260806 00:49:41.336570 30592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "9727470a859c47588c311b0f3a197546"
format_stamp: "Formatted at 2026-08-06 00:49:41 on dist-test-slave-45dx"
server_key: "0441ae91678b870d69c54081db79e157"
server_key_iv: "4e9ef44717f7b9e57bb1e1b39a1e97cb"
server_key_version: "encryptionkey@0"
I20260806 00:49:41.340196 30592 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:49:41.342730 30612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:41.343926 30592 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:49:41.344063 30592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "9727470a859c47588c311b0f3a197546"
format_stamp: "Formatted at 2026-08-06 00:49:41 on dist-test-slave-45dx"
server_key: "0441ae91678b870d69c54081db79e157"
server_key_iv: "4e9ef44717f7b9e57bb1e1b39a1e97cb"
server_key_version: "encryptionkey@0"
I20260806 00:49:41.344177 30592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:41.348613 30592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:41.349210 30592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:41.349393 30592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:41.349967 30592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:41.350898 30592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:41.350991 30592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:41.351061 30592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:41.351114 30592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:41.360590 30592 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:34735
I20260806 00:49:41.360661 30786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:34735 every 8 connection(s)
I20260806 00:49:41.361564 30592 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:49:41.368054 30171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:41.363475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:54981 (local address 127.15.147.190:46687)
0806 00:49:41.363626 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:41.363630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:41.364776 (+ 1146us) server_negotiation.cc:407] Connection header received
0806 00:49:41.366201 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:41.366205 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:41.366249 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:41.366362 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:41.367546 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:41.367549 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:41.367552 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:49:41.367564 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:41.367584 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:49:41.367601 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:41.367696 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:41.367855 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 00:49:41.367899 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 00:49:41.368896 30790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:41.363881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:46687 (local address 127.15.147.131:54981)
0806 00:49:41.364632 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:41.364673 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:41.365996 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:41.366530 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:41.366536 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:41.366656 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:41.367284 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:49:41.367286 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:41.367326 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:49:41.367399 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:41.367692 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:41.367701 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:41.367709 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 00:49:41.367893 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 00:49:41.368121 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":120,"threads_started":1}
I20260806 00:49:41.370045 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 30592
I20260806 00:49:41.370041 30787 heartbeater.cc:347] Connected to a master server at 127.15.147.190:46687
I20260806 00:49:41.370158 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:49:41.370270 30787 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:41.370508 15950 external_mini_cluster.cc:1428] Setting key 2e6b84bb4da1ad2743ef6aabf153cb7d
I20260806 00:49:41.370764 30787 heartbeater.cc:511] Master 127.15.147.190:46687 requested a full tablet report, sending...
I20260806 00:49:41.371845 30117 ts_manager.cc:194] Registered new tserver with Master: 9727470a859c47588c311b0f3a197546 (127.15.147.131:34735)
I20260806 00:49:41.372543 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 00:49:41.375499 30171 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:49:41.374469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52610 (local address 127.15.147.190:46687)
0806 00:49:41.374594 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:41.374601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:41.374829 (+ 228us) server_negotiation.cc:407] Connection header received
0806 00:49:41.375006 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:41.375009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:41.375062 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:41.375152 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:41.375398 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52610: BlockingRecv error: recv got EOF from 127.0.0.1:52610 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 00:49:41.375684 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 30206
I20260806 00:49:41.398978 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 30397
I20260806 00:49:41.419344 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 30592
I20260806 00:49:41.433945 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 30078
2026-08-06T00:49:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (886 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:49:41 dist-test-slave-45dx krb5kdc[30807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:49:41 dist-test-slave-45dx krb5kdc[30807](info): set up 2 sockets
Aug 06 00:49:41 dist-test-slave-45dx krb5kdc[30807](info): commencing operation
krb5kdc: starting...
W20260806 00:49:42.637673 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:49:42 dist-test-slave-45dx krb5kdc[30807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:49:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:49:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:49:42.761739 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:39393
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:38633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:39393
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:42.871423 30822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:42.871829 30822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:42.871903 30822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:42.875551 30822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:49:42.875626 30822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:42.875681 30822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:42.875718 30822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:49:42.875752 30822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:49:42.880818 30822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:38633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:39393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:39393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:42.881810 30822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:42.882788 30822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:42.891332 30829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:42.891332 30837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:42.891475 30822 server_base.cc:1034] running on GCE node
W20260806 00:49:42.891659 30830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:42.892249 30822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:42.893343 30822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:42.894510 30822 hybrid_clock.cc:648] HybridClock initialized: now 1785977382894491 us; error 31 us; skew 500 ppm
Aug 06 00:49:42 dist-test-slave-45dx krb5kdc[30807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977382, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:42.903111 30822 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:49:42.904029 30822 webserver.cc:460] Webserver started at http://127.15.147.190:38525/ using document root <none> and password file <none>
I20260806 00:49:42.904639 30822 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:42.904701 30822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:42.904928 30822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:42.906888 30822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "c7e89922e8cf43a69ec03e18e45eb2f4"
format_stamp: "Formatted at 2026-08-06 00:49:42 on dist-test-slave-45dx"
server_key: "b98cbb09715f8a9fb70eb60342dc1049"
server_key_iv: "2bb3fc5b69808d7b71cc80fefe35e2ef"
server_key_version: "encryptionkey@0"
I20260806 00:49:42.907449 30822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "c7e89922e8cf43a69ec03e18e45eb2f4"
format_stamp: "Formatted at 2026-08-06 00:49:42 on dist-test-slave-45dx"
server_key: "b98cbb09715f8a9fb70eb60342dc1049"
server_key_iv: "2bb3fc5b69808d7b71cc80fefe35e2ef"
server_key_version: "encryptionkey@0"
I20260806 00:49:42.911049 30822 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260806 00:49:42.913357 30851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:42.914525 30822 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:49:42.914649 30822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "c7e89922e8cf43a69ec03e18e45eb2f4"
format_stamp: "Formatted at 2026-08-06 00:49:42 on dist-test-slave-45dx"
server_key: "b98cbb09715f8a9fb70eb60342dc1049"
server_key_iv: "2bb3fc5b69808d7b71cc80fefe35e2ef"
server_key_version: "encryptionkey@0"
I20260806 00:49:42.914750 30822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:42.928113 30822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:42.931080 30822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:42.931288 30822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:42.938510 30822 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:39393
I20260806 00:49:42.938807 30920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:39393 every 8 connection(s)
I20260806 00:49:42.939484 30822 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:49:42.942314 30921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:42.947679 30921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4: Bootstrap starting.
I20260806 00:49:42.948025 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 30822
I20260806 00:49:42.948153 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:49:42.948487 15950 external_mini_cluster.cc:1428] Setting key 93a691235b75a0b59d249c2968f63a63
I20260806 00:49:42.950417 30921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:42.951125 30921 log.cc:824] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:42.953135 30921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4: No bootstrap required, opened a new log
Aug 06 00:49:42 dist-test-slave-45dx krb5kdc[30807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:49:42.957620 30921 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e89922e8cf43a69ec03e18e45eb2f4" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39393 } }
I20260806 00:49:42.957875 30921 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:42.957930 30921 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7e89922e8cf43a69ec03e18e45eb2f4, State: Initialized, Role: FOLLOWER
I20260806 00:49:42.958355 30921 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e89922e8cf43a69ec03e18e45eb2f4" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39393 } }
I20260806 00:49:42.958515 30921 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:49:42.958585 30921 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:49:42.958732 30921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:42.959578 30921 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e89922e8cf43a69ec03e18e45eb2f4" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39393 } }
I20260806 00:49:42.959899 30921 leader_election.cc:302] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7e89922e8cf43a69ec03e18e45eb2f4; no voters:
I20260806 00:49:42.960175 30921 leader_election.cc:288] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:49:42.960279 30926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:42.960547 30926 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4 [term 1 LEADER]: Becoming Leader. State: Replica: c7e89922e8cf43a69ec03e18e45eb2f4, State: Running, Role: LEADER
I20260806 00:49:42.960871 30926 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e89922e8cf43a69ec03e18e45eb2f4" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39393 } }
I20260806 00:49:42.961599 30921 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:49:42.963277 30929 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7e89922e8cf43a69ec03e18e45eb2f4. Latest consensus state: current_term: 1 leader_uuid: "c7e89922e8cf43a69ec03e18e45eb2f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e89922e8cf43a69ec03e18e45eb2f4" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39393 } } }
I20260806 00:49:42.963282 30928 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7e89922e8cf43a69ec03e18e45eb2f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e89922e8cf43a69ec03e18e45eb2f4" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39393 } } }
I20260806 00:49:42.963397 30929 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:42.963430 30928 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4 [sys.catalog]: This master's current role is: LEADER
I20260806 00:49:42.963663 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:42.949795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56586 (local address 127.15.147.190:39393)
0806 00:49:42.950193 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:42.950209 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:42.950243 (+ 34us) server_negotiation.cc:407] Connection header received
0806 00:49:42.950892 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:42.950909 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:42.951208 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:42.951545 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:42.952377 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:42.953378 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:42.954494 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:42.954944 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:42.955055 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:42.957992 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:42.957999 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:42.958015 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:42.958047 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:42.960022 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:42.960652 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:42.960658 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:42.960664 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:42.960796 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:42.961161 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:42.961164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:42.961166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:42.961669 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:42.961903 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:42.962358 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 00:49:42.962671 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":125,"threads_started":1}
I20260806 00:49:42.963749 30950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:49:42.967079 30950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:49:42.972342 30950 catalog_manager.cc:1347] Generated new cluster ID: d1231c9e64b34275a256d8993b49bcdd
I20260806 00:49:42.972409 30950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:49:42.987947 30950 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:49:42.989177 30950 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:49:42.998077 30950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4: Generated new TSK 0
I20260806 00:49:42.998776 30950 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
I20260806 00:49:43.031397 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39393
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:38633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:43.149610 30961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:43.149874 30961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:43.149943 30961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:43.153940 30961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:43.153997 30961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:43.154089 30961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:49:43.159242 30961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:38633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:43.160251 30961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:43.161298 30961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:43.171236 30970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:43.171206 30974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:43.171329 30961 server_base.cc:1034] running on GCE node
W20260806 00:49:43.171316 30976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:43.171913 30961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:43.172616 30961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:43.173806 30961 hybrid_clock.cc:648] HybridClock initialized: now 1785977383173782 us; error 34 us; skew 500 ppm
Aug 06 00:49:43 dist-test-slave-45dx krb5kdc[30807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977383, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:43.176532 30961 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:49:43.177282 30961 webserver.cc:460] Webserver started at http://127.15.147.129:45535/ using document root <none> and password file <none>
I20260806 00:49:43.177783 30961 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:43.177865 30961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:43.178119 30961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:43.180212 30961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "6065e279e9134fdb907db9c3da641953"
format_stamp: "Formatted at 2026-08-06 00:49:43 on dist-test-slave-45dx"
server_key: "9f8ed5c947355e5a283d8eae6eee7377"
server_key_iv: "9d9d7e3eea5e6581710600fdcbf101ac"
server_key_version: "encryptionkey@0"
I20260806 00:49:43.180898 30961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "6065e279e9134fdb907db9c3da641953"
format_stamp: "Formatted at 2026-08-06 00:49:43 on dist-test-slave-45dx"
server_key: "9f8ed5c947355e5a283d8eae6eee7377"
server_key_iv: "9d9d7e3eea5e6581710600fdcbf101ac"
server_key_version: "encryptionkey@0"
I20260806 00:49:43.184661 30961 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:49:43.187036 30984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:43.188239 30961 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:49:43.188414 30961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "6065e279e9134fdb907db9c3da641953"
format_stamp: "Formatted at 2026-08-06 00:49:43 on dist-test-slave-45dx"
server_key: "9f8ed5c947355e5a283d8eae6eee7377"
server_key_iv: "9d9d7e3eea5e6581710600fdcbf101ac"
server_key_version: "encryptionkey@0"
I20260806 00:49:43.188519 30961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:43.198081 30961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:43.200809 30961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:43.200961 30961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:43.201447 30961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:43.202188 30961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:43.202235 30961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:43.202271 30961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:43.202294 30961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:43.211320 30961 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:33479
I20260806 00:49:43.211357 31154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:33479 every 8 connection(s)
I20260806 00:49:43.212206 30961 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:49:43.218770 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 30961
I20260806 00:49:43.218890 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:49:43.219192 15950 external_mini_cluster.cc:1428] Setting key b5a4ffe36d1f74700217a48444c4595d
Aug 06 00:49:43 dist-test-slave-45dx krb5kdc[30807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977383, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:49:43.227010 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.214071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:40261 (local address 127.15.147.190:39393)
0806 00:49:43.214214 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:43.214218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:43.215089 (+ 871us) server_negotiation.cc:407] Connection header received
0806 00:49:43.216521 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:43.216524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:43.216573 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:43.216659 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:43.217807 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.218414 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.219909 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.220142 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.220368 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:43.223387 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:43.223390 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:43.223393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:43.223417 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:43.224973 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:43.225488 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:43.225493 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:43.225494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:43.225574 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:43.225922 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:43.225927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:43.225929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:43.226108 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:43.226213 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:43.226692 (+ 479us) server_negotiation.cc:299] Negotiation successful
0806 00:49:43.226829 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:49:43.227778 31157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.214432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:39393 (local address 127.15.147.129:40261)
0806 00:49:43.214906 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:43.214965 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:43.216265 (+ 1300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:43.216841 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:43.216847 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:43.217188 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:43.217663 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:43.217677 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.218599 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.218601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:43.219778 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:43.219788 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.220459 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.220464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:43.220684 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:43.221342 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:43.221359 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:43.223200 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:43.225095 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:43.225101 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:43.225114 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:43.225382 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:43.225668 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:43.225671 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:43.225673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:43.225823 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:43.226213 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:43.226221 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:43.226460 (+ 239us) client_negotiation.cc:769] Sending connection context
0806 00:49:43.226641 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 00:49:43.226952 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":123,"threads_started":1}
I20260806 00:49:43.229414 31155 heartbeater.cc:347] Connected to a master server at 127.15.147.190:39393
I20260806 00:49:43.229684 31155 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.147.130@KRBTEST.COM; defaulting to no policy
I20260806 00:49:43.230206 31155 heartbeater.cc:511] Master 127.15.147.190:39393 requested a full tablet report, sending...
I20260806 00:49:43.231808 30873 ts_manager.cc:194] Registered new tserver with Master: 6065e279e9134fdb907db9c3da641953 (127.15.147.129:33479)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.130@KRBTEST.COM" created.
I20260806 00:49:43.233127 30873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:40261
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
I20260806 00:49:43.249445 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39393
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:38633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:43.361933 31161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:43.362170 31161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:43.362213 31161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:43.366304 31161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:43.366364 31161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:43.366422 31161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:49:43.371507 31161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:38633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:43.372488 31161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:43.373466 31161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:43.383297 31170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:43.383353 31167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:43.383370 31161 server_base.cc:1034] running on GCE node
W20260806 00:49:43.383318 31168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:43.384040 31161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:43.384743 31161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:43.385915 31161 hybrid_clock.cc:648] HybridClock initialized: now 1785977383385892 us; error 33 us; skew 500 ppm
Aug 06 00:49:43 dist-test-slave-45dx krb5kdc[30807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977383, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:43.388914 31161 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:49:43.389827 31161 webserver.cc:460] Webserver started at http://127.15.147.130:35561/ using document root <none> and password file <none>
I20260806 00:49:43.390390 31161 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:43.390452 31161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:43.390710 31161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:43.392989 31161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "3a64f3a49036431c92986cf5a683f272"
format_stamp: "Formatted at 2026-08-06 00:49:43 on dist-test-slave-45dx"
server_key: "332e6d67d2095e0ddb138aa804981091"
server_key_iv: "851de41e1938021d6bb44b10b5d25ed2"
server_key_version: "encryptionkey@0"
I20260806 00:49:43.393553 31161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "3a64f3a49036431c92986cf5a683f272"
format_stamp: "Formatted at 2026-08-06 00:49:43 on dist-test-slave-45dx"
server_key: "332e6d67d2095e0ddb138aa804981091"
server_key_iv: "851de41e1938021d6bb44b10b5d25ed2"
server_key_version: "encryptionkey@0"
I20260806 00:49:43.397342 31161 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:49:43.399717 31179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:43.400992 31161 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:49:43.401132 31161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "3a64f3a49036431c92986cf5a683f272"
format_stamp: "Formatted at 2026-08-06 00:49:43 on dist-test-slave-45dx"
server_key: "332e6d67d2095e0ddb138aa804981091"
server_key_iv: "851de41e1938021d6bb44b10b5d25ed2"
server_key_version: "encryptionkey@0"
I20260806 00:49:43.401244 31161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:43.415684 31161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:43.419132 31161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:43.419323 31161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:43.419894 31161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:43.420945 31161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:43.421017 31161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:43.421092 31161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:43.421151 31161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:43.430493 31161 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:38413
I20260806 00:49:43.430526 31348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:38413 every 8 connection(s)
I20260806 00:49:43.431435 31161 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:49:43.435930 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 31161
I20260806 00:49:43.436076 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:49:43.436481 15950 external_mini_cluster.cc:1428] Setting key 1904474df8237427f139a0822eb23abb
Aug 06 00:49:43 dist-test-slave-45dx krb5kdc[30807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977383, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:49:43.447652 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.433474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:34545 (local address 127.15.147.190:39393)
0806 00:49:43.433632 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:43.433635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:43.434468 (+ 833us) server_negotiation.cc:407] Connection header received
0806 00:49:43.436120 (+ 1652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:43.436126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:43.436199 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:43.436283 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:43.437877 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.438510 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.440295 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.440734 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.440868 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:43.443669 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:43.443676 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:43.443682 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:43.443720 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:43.445526 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:43.446039 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:43.446045 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:43.446049 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:43.446144 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:43.446493 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:43.446496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:43.446498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:43.446744 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:43.446852 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:43.447334 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 00:49:43.447474 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
WARNING: no policy specified for kudu/127.15.147.131@KRBTEST.COM; defaulting to no policy
I20260806 00:49:43.448498 31356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.433842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:39393 (local address 127.15.147.130:34545)
0806 00:49:43.434312 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:43.434356 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:43.435805 (+ 1449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:43.436559 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:43.436570 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:43.437052 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:43.437679 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:43.437700 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.438654 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.438660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:43.440153 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:43.440169 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.440906 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.440909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:43.441056 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:43.441729 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:43.441746 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:43.443488 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:43.445646 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:43.445652 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:43.445664 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:43.445931 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:43.446242 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:43.446244 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:43.446246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:43.446394 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:43.446850 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:43.446855 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:43.447092 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 00:49:43.447245 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 00:49:43.447506 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":131,"threads_started":1}
I20260806 00:49:43.449833 31351 heartbeater.cc:347] Connected to a master server at 127.15.147.190:39393
I20260806 00:49:43.450095 31351 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:43.450610 31351 heartbeater.cc:511] Master 127.15.147.190:39393 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.131@KRBTEST.COM" created.
I20260806 00:49:43.451756 30873 ts_manager.cc:194] Registered new tserver with Master: 3a64f3a49036431c92986cf5a683f272 (127.15.147.130:38413)
I20260806 00:49:43.452467 30873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:34545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
I20260806 00:49:43.467687 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39393
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:38633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:49:43.581131 31364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:49:43.581377 31364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:49:43.581457 31364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:49:43.585430 31364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:49:43.585480 31364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:49:43.585578 31364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:49:43.590745 31364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:38633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:49:43.591822 31364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:49:43.592875 31364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:49:43.602314 31372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:43.602340 31369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:49:43.602353 31374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:49:43.602408 31364 server_base.cc:1034] running on GCE node
I20260806 00:49:43.602933 31364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:49:43.603534 31364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:49:43.604719 31364 hybrid_clock.cc:648] HybridClock initialized: now 1785977383604710 us; error 31 us; skew 500 ppm
Aug 06 00:49:43 dist-test-slave-45dx krb5kdc[30807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977383, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:49:43.607448 31364 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:49:43.608233 31364 webserver.cc:460] Webserver started at http://127.15.147.131:46287/ using document root <none> and password file <none>
I20260806 00:49:43.608790 31364 fs_manager.cc:359] Metadata directory not provided
I20260806 00:49:43.608855 31364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:49:43.609014 31364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:49:43.610893 31364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "8331c25ef23846f9bba99bc76f89ce34"
format_stamp: "Formatted at 2026-08-06 00:49:43 on dist-test-slave-45dx"
server_key: "7dfc6f74f7c8d3b7ead8c747e44b9f0a"
server_key_iv: "de37fd65815d706db224a28c5c6dc86b"
server_key_version: "encryptionkey@0"
I20260806 00:49:43.611424 31364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "8331c25ef23846f9bba99bc76f89ce34"
format_stamp: "Formatted at 2026-08-06 00:49:43 on dist-test-slave-45dx"
server_key: "7dfc6f74f7c8d3b7ead8c747e44b9f0a"
server_key_iv: "de37fd65815d706db224a28c5c6dc86b"
server_key_version: "encryptionkey@0"
I20260806 00:49:43.615087 31364 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 00:49:43.617489 31388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:43.618680 31364 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:49:43.618773 31364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "8331c25ef23846f9bba99bc76f89ce34"
format_stamp: "Formatted at 2026-08-06 00:49:43 on dist-test-slave-45dx"
server_key: "7dfc6f74f7c8d3b7ead8c747e44b9f0a"
server_key_iv: "de37fd65815d706db224a28c5c6dc86b"
server_key_version: "encryptionkey@0"
I20260806 00:49:43.618853 31364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:49:43.631544 31364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:49:43.634475 31364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:49:43.634625 31364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:49:43.635142 31364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:49:43.635903 31364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:49:43.635950 31364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:43.635989 31364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:49:43.636004 31364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:49:43.645182 31364 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:45113
I20260806 00:49:43.645224 31562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:45113 every 8 connection(s)
I20260806 00:49:43.646054 31364 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:49:43.654186 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 31364
I20260806 00:49:43.654335 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:49:43.654681 15950 external_mini_cluster.cc:1428] Setting key 57d6455edde2f99dc0f2ed6dce61b520
Aug 06 00:49:43 dist-test-slave-45dx krb5kdc[30807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977383, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:49:43.659629 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.647799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:55633 (local address 127.15.147.190:39393)
0806 00:49:43.647944 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:43.647947 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:43.648800 (+ 853us) server_negotiation.cc:407] Connection header received
0806 00:49:43.650192 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:43.650196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:43.650256 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:43.650334 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:43.651482 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.652024 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.653198 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.653429 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.653563 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:43.656213 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:43.656217 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:43.656219 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:43.656242 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:43.657632 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:43.658147 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:43.658152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:43.658155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:43.658236 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:43.658579 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:43.658584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:43.658586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:43.658762 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:43.658874 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:43.659344 (+ 470us) server_negotiation.cc:299] Negotiation successful
0806 00:49:43.659463 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:49:43.660527 31565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.648151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:39393 (local address 127.15.147.131:55633)
0806 00:49:43.648649 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:43.648690 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:43.649976 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:43.650505 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:43.650512 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:43.650857 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:49:43.651322 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:43.651344 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.652163 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.652169 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:43.653048 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:43.653057 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.653607 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.653611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:43.653781 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:43.654435 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:49:43.654458 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:49:43.656045 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:49:43.657757 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:43.657762 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:43.657774 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:43.658041 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:43.658328 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:49:43.658332 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:49:43.658334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:49:43.658480 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:49:43.658876 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:49:43.658885 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:49:43.659111 (+ 226us) client_negotiation.cc:769] Sending connection context
0806 00:49:43.659294 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 00:49:43.659594 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":187,"threads_started":1}
I20260806 00:49:43.661585 31563 heartbeater.cc:347] Connected to a master server at 127.15.147.190:39393
I20260806 00:49:43.661808 31563 heartbeater.cc:464] Registering TS with master...
I20260806 00:49:43.662310 31563 heartbeater.cc:511] Master 127.15.147.190:39393 requested a full tablet report, sending...
I20260806 00:49:43.663269 30873 ts_manager.cc:194] Registered new tserver with Master: 8331c25ef23846f9bba99bc76f89ce34 (127.15.147.131:45113)
I20260806 00:49:43.663862 30873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:55633
I20260806 00:49:43.668742 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:49:43.678468 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.670677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56594 (local address 127.15.147.190:39393)
0806 00:49:43.670838 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:43.670843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:43.671057 (+ 214us) server_negotiation.cc:407] Connection header received
0806 00:49:43.671206 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:43.671210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:43.671262 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:43.671349 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:49:43.672101 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.672688 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.673578 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.673799 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.673940 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:43.675374 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:49:43.675378 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:49:43.675381 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:49:43.675411 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:49:43.676698 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:43.677135 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:43.677139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:43.677141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:43.677231 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:49:43.677525 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:49:43.677529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:49:43.677531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:49:43.677764 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:49:43.677927 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:43.678168 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 00:49:43.678288 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:49:43.681442 30873 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56594:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:49:43.684732 30873 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:49:43.704075 31593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.698130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51436 (local address 127.15.147.131:45113)
0806 00:49:43.698615 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:43.698637 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:43.698662 (+ 25us) server_negotiation.cc:407] Connection header received
0806 00:49:43.698863 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:43.698872 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:43.699105 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:43.699273 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:43.699640 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.700961 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.702844 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.703430 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.703560 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:43.703696 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:43.703819 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 00:49:43.703921 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":172,"threads_started":1}
I20260806 00:49:43.704074 31588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.697769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:38413 (local address 127.0.0.1:50748)
0806 00:49:43.698829 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:43.698843 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:43.698921 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:43.699544 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:43.699551 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:43.699575 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:43.699696 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:43.699702 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.701144 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.701148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:43.702519 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:43.702531 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.703446 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.703450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:43.703575 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:43.703615 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 00:49:43.703727 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 00:49:43.703875 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":987,"mutex_wait_us":95,"thread_start_us":101,"threads_started":1}
I20260806 00:49:43.704070 31586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.697467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:45113 (local address 127.0.0.1:51436)
0806 00:49:43.698448 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:43.698510 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:43.698650 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:43.699282 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:43.699288 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:43.699309 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:43.699496 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:43.699506 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.701144 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.701148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:43.702684 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:43.702696 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.703565 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.703569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:43.703662 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:43.703686 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 00:49:43.703778 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 00:49:43.703883 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":103,"thread_start_us":116,"threads_started":1}
I20260806 00:49:43.704448 31594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.698268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50748 (local address 127.15.147.130:38413)
0806 00:49:43.699019 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:43.699035 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:43.699059 (+ 24us) server_negotiation.cc:407] Connection header received
0806 00:49:43.699140 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:43.699144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:43.699336 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:43.699535 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:43.699821 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.700985 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.702659 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.703305 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.703409 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:43.703559 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:43.704147 (+ 588us) server_negotiation.cc:299] Negotiation successful
0806 00:49:43.704264 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":109,"threads_started":1}
I20260806 00:49:43.705416 31590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.697410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:33479 (local address 127.0.0.1:60550)
0806 00:49:43.698613 (+ 1203us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:43.698628 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:43.698720 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:43.699646 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:43.699649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:43.699664 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:43.699735 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:43.699744 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.700946 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.700951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:43.702519 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:43.702531 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.704998 (+ 2467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.705002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:43.705106 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:43.705132 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:49:43.705204 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 00:49:43.705266 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":68,"thread_start_us":124,"threads_started":1}
I20260806 00:49:43.706248 31589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.697699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60550 (local address 127.15.147.129:33479)
0806 00:49:43.699167 (+ 1468us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:43.699184 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:43.699209 (+ 25us) server_negotiation.cc:407] Connection header received
0806 00:49:43.699285 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:43.699290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:43.699471 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:43.699887 (+ 416us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:43.700015 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.700801 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.703623 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.704858 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.705836 (+ 978us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:43.705960 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:43.706012 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 00:49:43.706102 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1350,"thread_start_us":128,"threads_started":1}
I20260806 00:49:43.707526 31464 tablet_service.cc:1467] Processing CreateTablet for tablet e01a573dccb643939bf5c6d77ae6808d (DEFAULT_TABLE table=test-table [id=acb0e86654d34e5db517d2a2cf407627]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:43.707547 31245 tablet_service.cc:1467] Processing CreateTablet for tablet e01a573dccb643939bf5c6d77ae6808d (DEFAULT_TABLE table=test-table [id=acb0e86654d34e5db517d2a2cf407627]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:43.708626 31464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e01a573dccb643939bf5c6d77ae6808d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:43.708660 31245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e01a573dccb643939bf5c6d77ae6808d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:43.708917 31057 tablet_service.cc:1467] Processing CreateTablet for tablet e01a573dccb643939bf5c6d77ae6808d (DEFAULT_TABLE table=test-table [id=acb0e86654d34e5db517d2a2cf407627]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:49:43.709867 31057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e01a573dccb643939bf5c6d77ae6808d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:49:43.714958 31595 tablet_bootstrap.cc:492] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34: Bootstrap starting.
I20260806 00:49:43.716625 31601 tablet_bootstrap.cc:492] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272: Bootstrap starting.
I20260806 00:49:43.716640 31602 tablet_bootstrap.cc:492] T e01a573dccb643939bf5c6d77ae6808d P 6065e279e9134fdb907db9c3da641953: Bootstrap starting.
I20260806 00:49:43.717471 31595 tablet_bootstrap.cc:654] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:43.718931 31602 tablet_bootstrap.cc:654] T e01a573dccb643939bf5c6d77ae6808d P 6065e279e9134fdb907db9c3da641953: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:43.718918 31595 log.cc:824] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:43.719205 31601 tablet_bootstrap.cc:654] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272: Neither blocks nor log segments found. Creating new log.
I20260806 00:49:43.720049 31601 log.cc:824] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:43.720228 31602 log.cc:824] T e01a573dccb643939bf5c6d77ae6808d P 6065e279e9134fdb907db9c3da641953: Log is configured to *not* fsync() on all Append() calls
I20260806 00:49:43.721657 31595 tablet_bootstrap.cc:492] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34: No bootstrap required, opened a new log
I20260806 00:49:43.721879 31595 ts_tablet_manager.cc:1397] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 00:49:43.722085 31601 tablet_bootstrap.cc:492] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272: No bootstrap required, opened a new log
I20260806 00:49:43.722267 31601 ts_tablet_manager.cc:1397] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:49:43.722383 31602 tablet_bootstrap.cc:492] T e01a573dccb643939bf5c6d77ae6808d P 6065e279e9134fdb907db9c3da641953: No bootstrap required, opened a new log
I20260806 00:49:43.722558 31602 ts_tablet_manager.cc:1397] T e01a573dccb643939bf5c6d77ae6808d P 6065e279e9134fdb907db9c3da641953: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:49:43.725046 31595 raft_consensus.cc:348] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6065e279e9134fdb907db9c3da641953" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33479 } } peers { permanent_uuid: "8331c25ef23846f9bba99bc76f89ce34" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45113 } } peers { permanent_uuid: "3a64f3a49036431c92986cf5a683f272" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 38413 } }
I20260806 00:49:43.725287 31595 raft_consensus.cc:374] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:43.725353 31595 raft_consensus.cc:729] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8331c25ef23846f9bba99bc76f89ce34, State: Initialized, Role: FOLLOWER
I20260806 00:49:43.725805 31595 consensus_queue.cc:260] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6065e279e9134fdb907db9c3da641953" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33479 } } peers { permanent_uuid: "8331c25ef23846f9bba99bc76f89ce34" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45113 } } peers { permanent_uuid: "3a64f3a49036431c92986cf5a683f272" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 38413 } }
I20260806 00:49:43.726125 31601 raft_consensus.cc:348] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6065e279e9134fdb907db9c3da641953" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33479 } } peers { permanent_uuid: "8331c25ef23846f9bba99bc76f89ce34" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45113 } } peers { permanent_uuid: "3a64f3a49036431c92986cf5a683f272" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 38413 } }
I20260806 00:49:43.726372 31601 raft_consensus.cc:374] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:43.726423 31601 raft_consensus.cc:729] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a64f3a49036431c92986cf5a683f272, State: Initialized, Role: FOLLOWER
I20260806 00:49:43.726466 31602 raft_consensus.cc:348] T e01a573dccb643939bf5c6d77ae6808d P 6065e279e9134fdb907db9c3da641953 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6065e279e9134fdb907db9c3da641953" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33479 } } peers { permanent_uuid: "8331c25ef23846f9bba99bc76f89ce34" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45113 } } peers { permanent_uuid: "3a64f3a49036431c92986cf5a683f272" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 38413 } }
I20260806 00:49:43.726691 31602 raft_consensus.cc:374] T e01a573dccb643939bf5c6d77ae6808d P 6065e279e9134fdb907db9c3da641953 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:49:43.726739 31602 raft_consensus.cc:729] T e01a573dccb643939bf5c6d77ae6808d P 6065e279e9134fdb907db9c3da641953 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6065e279e9134fdb907db9c3da641953, State: Initialized, Role: FOLLOWER
I20260806 00:49:43.726862 31595 ts_tablet_manager.cc:1428] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:49:43.726854 31601 consensus_queue.cc:260] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6065e279e9134fdb907db9c3da641953" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33479 } } peers { permanent_uuid: "8331c25ef23846f9bba99bc76f89ce34" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45113 } } peers { permanent_uuid: "3a64f3a49036431c92986cf5a683f272" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 38413 } }
I20260806 00:49:43.727259 31563 heartbeater.cc:503] Master 127.15.147.190:39393 was elected leader, sending a full tablet report...
I20260806 00:49:43.727195 31602 consensus_queue.cc:260] T e01a573dccb643939bf5c6d77ae6808d P 6065e279e9134fdb907db9c3da641953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6065e279e9134fdb907db9c3da641953" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33479 } } peers { permanent_uuid: "8331c25ef23846f9bba99bc76f89ce34" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45113 } } peers { permanent_uuid: "3a64f3a49036431c92986cf5a683f272" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 38413 } }
I20260806 00:49:43.727768 31601 ts_tablet_manager.cc:1428] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 00:49:43.727869 31155 heartbeater.cc:503] Master 127.15.147.190:39393 was elected leader, sending a full tablet report...
I20260806 00:49:43.728073 31351 heartbeater.cc:503] Master 127.15.147.190:39393 was elected leader, sending a full tablet report...
I20260806 00:49:43.728111 31602 ts_tablet_manager.cc:1428] T e01a573dccb643939bf5c6d77ae6808d P 6065e279e9134fdb907db9c3da641953: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:49:43.872475 31606 raft_consensus.cc:482] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:49:43.872713 31606 raft_consensus.cc:504] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6065e279e9134fdb907db9c3da641953" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33479 } } peers { permanent_uuid: "8331c25ef23846f9bba99bc76f89ce34" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45113 } } peers { permanent_uuid: "3a64f3a49036431c92986cf5a683f272" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 38413 } }
I20260806 00:49:43.873942 31606 leader_election.cc:288] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6065e279e9134fdb907db9c3da641953 (127.15.147.129:33479), 3a64f3a49036431c92986cf5a683f272 (127.15.147.130:38413)
I20260806 00:49:43.878248 31589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.874314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:57233 (local address 127.15.147.129:33479)
0806 00:49:43.874475 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:43.874479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:43.874504 (+ 25us) server_negotiation.cc:407] Connection header received
0806 00:49:43.874705 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:43.874708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:43.874779 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:43.874869 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:43.875199 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.875757 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.877180 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.877585 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.877688 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:43.877791 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:43.878022 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 00:49:43.878117 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 00:49:43.878254 31565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.874208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:33479 (local address 127.15.147.131:57233)
0806 00:49:43.874438 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:43.874455 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:43.874565 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:43.874903 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:43.874905 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:43.874925 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:43.875074 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:43.875081 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.875915 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.875920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:43.876987 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:43.877000 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.877734 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.877738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:43.877867 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:43.877898 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 00:49:43.877967 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 00:49:43.878081 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":139}
I20260806 00:49:43.878288 31594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.874370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:42281 (local address 127.15.147.130:38413)
0806 00:49:43.874504 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:43.874508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:43.874820 (+ 312us) server_negotiation.cc:407] Connection header received
0806 00:49:43.874915 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:43.874918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:43.874967 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:43.875050 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:49:43.875369 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.875864 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.877149 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.877612 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.877773 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:43.877830 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:43.878068 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 00:49:43.878158 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:49:43.878264 31609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.874246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:38413 (local address 127.15.147.131:42281)
0806 00:49:43.874706 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:43.874719 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:49:43.874828 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:49:43.875071 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:49:43.875073 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:49:43.875087 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:49:43.875243 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:43.875250 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.875987 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.875990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:43.876979 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:49:43.876991 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.877790 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.877794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:49:43.877902 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:43.877921 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 00:49:43.878018 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 00:49:43.878095 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":1,"thread_start_us":128,"threads_started":1}
I20260806 00:49:43.879096 31085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01a573dccb643939bf5c6d77ae6808d" candidate_uuid: "8331c25ef23846f9bba99bc76f89ce34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6065e279e9134fdb907db9c3da641953" is_pre_election: true
I20260806 00:49:43.879161 31274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01a573dccb643939bf5c6d77ae6808d" candidate_uuid: "8331c25ef23846f9bba99bc76f89ce34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a64f3a49036431c92986cf5a683f272" is_pre_election: true
I20260806 00:49:43.879454 31085 raft_consensus.cc:2454] T e01a573dccb643939bf5c6d77ae6808d P 6065e279e9134fdb907db9c3da641953 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8331c25ef23846f9bba99bc76f89ce34 in term 0.
I20260806 00:49:43.879472 31274 raft_consensus.cc:2454] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8331c25ef23846f9bba99bc76f89ce34 in term 0.
I20260806 00:49:43.880043 31394 leader_election.cc:302] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3a64f3a49036431c92986cf5a683f272, 6065e279e9134fdb907db9c3da641953, 8331c25ef23846f9bba99bc76f89ce34; no voters:
I20260806 00:49:43.880366 31606 raft_consensus.cc:2781] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:49:43.880445 31606 raft_consensus.cc:482] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:49:43.880477 31606 raft_consensus.cc:3037] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:43.881599 31606 raft_consensus.cc:504] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6065e279e9134fdb907db9c3da641953" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33479 } } peers { permanent_uuid: "8331c25ef23846f9bba99bc76f89ce34" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45113 } } peers { permanent_uuid: "3a64f3a49036431c92986cf5a683f272" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 38413 } }
I20260806 00:49:43.882007 31606 leader_election.cc:288] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [CANDIDATE]: Term 1 election: Requested vote from peers 6065e279e9134fdb907db9c3da641953 (127.15.147.129:33479), 3a64f3a49036431c92986cf5a683f272 (127.15.147.130:38413)
I20260806 00:49:43.882426 31274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01a573dccb643939bf5c6d77ae6808d" candidate_uuid: "8331c25ef23846f9bba99bc76f89ce34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a64f3a49036431c92986cf5a683f272"
I20260806 00:49:43.882542 31274 raft_consensus.cc:3037] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:43.883889 31274 raft_consensus.cc:2454] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8331c25ef23846f9bba99bc76f89ce34 in term 1.
I20260806 00:49:43.884204 31085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e01a573dccb643939bf5c6d77ae6808d" candidate_uuid: "8331c25ef23846f9bba99bc76f89ce34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6065e279e9134fdb907db9c3da641953"
I20260806 00:49:43.884254 31393 leader_election.cc:302] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a64f3a49036431c92986cf5a683f272, 8331c25ef23846f9bba99bc76f89ce34; no voters:
I20260806 00:49:43.884409 31085 raft_consensus.cc:3037] T e01a573dccb643939bf5c6d77ae6808d P 6065e279e9134fdb907db9c3da641953 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:49:43.884469 31606 raft_consensus.cc:2781] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:49:43.884729 31606 raft_consensus.cc:686] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [term 1 LEADER]: Becoming Leader. State: Replica: 8331c25ef23846f9bba99bc76f89ce34, State: Running, Role: LEADER
I20260806 00:49:43.885116 31606 consensus_queue.cc:237] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6065e279e9134fdb907db9c3da641953" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33479 } } peers { permanent_uuid: "8331c25ef23846f9bba99bc76f89ce34" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45113 } } peers { permanent_uuid: "3a64f3a49036431c92986cf5a683f272" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 38413 } }
I20260806 00:49:43.885628 31085 raft_consensus.cc:2454] T e01a573dccb643939bf5c6d77ae6808d P 6065e279e9134fdb907db9c3da641953 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8331c25ef23846f9bba99bc76f89ce34 in term 1.
I20260806 00:49:43.888471 30870 catalog_manager.cc:5495] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8331c25ef23846f9bba99bc76f89ce34 (127.15.147.131). New cstate: current_term: 1 leader_uuid: "8331c25ef23846f9bba99bc76f89ce34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6065e279e9134fdb907db9c3da641953" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8331c25ef23846f9bba99bc76f89ce34" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a64f3a49036431c92986cf5a683f272" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 38413 } health_report { overall_health: UNKNOWN } } }
W20260806 00:49:43.896931 31564 tablet.cc:2369] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:49:43.932727 31352 tablet.cc:2369] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:49:43.939112 31593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.935296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51440 (local address 127.15.147.131:45113)
0806 00:49:43.935493 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 00:49:43.935497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:49:43.935512 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:49:43.935596 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:49:43.935598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:49:43.935671 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:49:43.935766 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:49:43.936020 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.936551 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.937447 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:49:43.937694 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:49:43.937788 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:49:43.938102 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:49:43.938702 (+ 600us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:49:43.938867 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 00:49:43.938904 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 00:49:43.938973 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260806 00:49:43.945912 31085 raft_consensus.cc:1261] T e01a573dccb643939bf5c6d77ae6808d P 6065e279e9134fdb907db9c3da641953 [term 1 FOLLOWER]: Refusing update from remote peer 8331c25ef23846f9bba99bc76f89ce34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:43.945922 31274 raft_consensus.cc:1261] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272 [term 1 FOLLOWER]: Refusing update from remote peer 8331c25ef23846f9bba99bc76f89ce34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:49:43.946851 31612 consensus_queue.cc:1035] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a64f3a49036431c92986cf5a683f272" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 38413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:43.947108 31606 consensus_queue.cc:1035] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6065e279e9134fdb907db9c3da641953" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 33479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:49:43.957383 31618 mvcc.cc:205] Tried to move back new op lower bound from 7315363364634320896 to 7315363364396728320. Current Snapshot: MvccSnapshot[applied={T|T < 7315363364634320896}]
I20260806 00:49:43.959313 31616 mvcc.cc:205] Tried to move back new op lower bound from 7315363364634320896 to 7315363364396728320. Current Snapshot: MvccSnapshot[applied={T|T < 7315363364634320896}]
I20260806 00:49:43.964989 30870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:49:43.965183 30870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:49:43.967592 30870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7e89922e8cf43a69ec03e18e45eb2f4: Sending DeleteTablet for 3 replicas of tablet e01a573dccb643939bf5c6d77ae6808d
I20260806 00:49:43.968482 31464 tablet_service.cc:1514] Processing DeleteTablet for tablet e01a573dccb643939bf5c6d77ae6808d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:43 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:51436
I20260806 00:49:43.968483 31245 tablet_service.cc:1514] Processing DeleteTablet for tablet e01a573dccb643939bf5c6d77ae6808d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:43 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:50748
I20260806 00:49:43.968495 31057 tablet_service.cc:1514] Processing DeleteTablet for tablet e01a573dccb643939bf5c6d77ae6808d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:49:43 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:60550
I20260806 00:49:43.969066 31633 tablet_replica.cc:330] T e01a573dccb643939bf5c6d77ae6808d P 6065e279e9134fdb907db9c3da641953: stopping tablet replica
I20260806 00:49:43.969214 31634 tablet_replica.cc:330] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34: stopping tablet replica
I20260806 00:49:43.969424 31633 raft_consensus.cc:2229] T e01a573dccb643939bf5c6d77ae6808d P 6065e279e9134fdb907db9c3da641953 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:43.969444 31634 raft_consensus.cc:2229] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:49:43.969735 31633 raft_consensus.cc:2258] T e01a573dccb643939bf5c6d77ae6808d P 6065e279e9134fdb907db9c3da641953 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:43.969755 31634 raft_consensus.cc:2258] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:43.970170 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 30961
I20260806 00:49:43.970890 31632 tablet_replica.cc:330] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272: stopping tablet replica
I20260806 00:49:43.971164 31632 raft_consensus.cc:2229] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:49:43.971360 31634 ts_tablet_manager.cc:1905] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:43.971444 31632 raft_consensus.cc:2258] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:49:43.973894 31632 ts_tablet_manager.cc:1905] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:49:43.974469 31634 ts_tablet_manager.cc:1918] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:49:43.974558 31634 log.cc:1196] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/e01a573dccb643939bf5c6d77ae6808d
I20260806 00:49:43.974900 31634 ts_tablet_manager.cc:1939] T e01a573dccb643939bf5c6d77ae6808d P 8331c25ef23846f9bba99bc76f89ce34: Deleting consensus metadata
I20260806 00:49:43.976169 30852 catalog_manager.cc:4841] TS 8331c25ef23846f9bba99bc76f89ce34 (127.15.147.131:45113): tablet e01a573dccb643939bf5c6d77ae6808d (table test-table [id=acb0e86654d34e5db517d2a2cf407627]) successfully deleted
I20260806 00:49:43.976811 31632 ts_tablet_manager.cc:1918] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:49:43.976886 31632 log.cc:1196] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/e01a573dccb643939bf5c6d77ae6808d
I20260806 00:49:43.977110 31632 ts_tablet_manager.cc:1939] T e01a573dccb643939bf5c6d77ae6808d P 3a64f3a49036431c92986cf5a683f272: Deleting consensus metadata
I20260806 00:49:43.979641 30854 catalog_manager.cc:4841] TS 3a64f3a49036431c92986cf5a683f272 (127.15.147.130:38413): tablet e01a573dccb643939bf5c6d77ae6808d (table test-table [id=acb0e86654d34e5db517d2a2cf407627]) successfully deleted
W20260806 00:49:43.988762 30855 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.147.129:33479 (error 108)
I20260806 00:49:43.988873 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 31161
I20260806 00:49:43.991401 31590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:49:43.991065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:33479 (local address 127.0.0.1:60558)
0806 00:49:43.991247 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 00:49:43.991311 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260806 00:49:43.991568 30855 catalog_manager.cc:4584] TS 6065e279e9134fdb907db9c3da641953 (127.15.147.129:33479): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e01a573dccb643939bf5c6d77ae6808d: Network error: Client connection negotiation failed: client connection to 127.15.147.129:33479: connect: Connection refused (error 111)
I20260806 00:49:44.001190 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 31364
I20260806 00:49:44.015267 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 30822
2026-08-06T00:49:44Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2584 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:49:44 dist-test-slave-45dx krb5kdc[31640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:49:44 dist-test-slave-45dx krb5kdc[31640](info): set up 2 sockets
Aug 06 00:49:44 dist-test-slave-45dx krb5kdc[31640](info): commencing operation
krb5kdc: starting...
W20260806 00:49:45.213779 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:49:45 dist-test-slave-45dx krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:49:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:49:45Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.147.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.147.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.147.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/rangeradmin_127.15.147.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.147.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/rangeradmin_127.15.147.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.147.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.147.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.147.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/rangerlookup_127.15.147.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.147.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/rangerlookup_127.15.147.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.147.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.147.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.147.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/HTTP_127.15.147.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.147.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/HTTP_127.15.147.148@KRBTEST.COM.keytab.
I20260806 00:49:45.387614 15950 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task3y4EW9/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/postgres -l logfile start
2026-08-06 00:49:47.921 UTC [31682] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 00:49:47.921 UTC [31682] LOG: listening on IPv4 address "127.15.147.148", port 50319
2026-08-06 00:49:47.925 UTC [31682] LOG: listening on Unix socket "/tmp/.s.PGSQL.50319"
2026-08-06 00:49:47.947 UTC [31685] LOG: database system was shut down at 2026-08-06 00:49:46 UTC
2026-08-06 00:49:47.950 UTC [31682] LOG: database system is ready to accept connections
I20260806 00:49:49.080018 15950 mini_postgres.cc:96] Postgres bound to 50319
2026-08-06 00:49:49.088 UTC [31696] FATAL: database "slave" does not exist
127.15.147.148:50319 - accepting connections
I20260806 00:49:49.089599 15950 mini_ranger.cc:161] Starting Ranger...
I20260806 00:49:49.107949 15950 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 00:49:49.623452 15950 mini_ranger.cc:87] Created ranger Postgres database
I20260806 00:49:49.623567 15950 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ranger-admin
2026-08-06 00:49:49,987 [I] DB FLAVOR :POSTGRES
2026-08-06 00:49:49,987 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 00:49:49,987 [I] Checking connection..
2026-08-06 00:49:49,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 00:49:50,252 [I] Checking connection passed.
2026-08-06 00:49:50,252 [I] --------- Verifying version history table ---------
2026-08-06 00:49:50,252 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 00:49:50,520 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 00:49:50,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 00:49:50,792 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 00:49:50,792 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 00:49:50,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 00:49:51.051 UTC [31801] WARNING: there is no transaction in progress
2026-08-06 00:49:51,060 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 00:49:51,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 00:49:51,328 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 00:49:51,328 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 00:49:51,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 00:49:51,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 00:49:51,842 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-45dx.c.gcp-upstream.internal','N') ;"
2026-08-06 00:49:52,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 00:49:52,354 [I] Table x_portal_user does not exist in database ranger
2026-08-06 00:49:52,355 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 00:49:52,619 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 00:49:52,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 00:49:52,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 00:49:53,136 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 00:49:53,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 00:49:54.102 UTC [31955] WARNING: there is no transaction in progress
2026-08-06 00:49:54.118 UTC [31955] WARNING: there is no transaction in progress
2026-08-06 00:49:54.133 UTC [31955] WARNING: there is no transaction in progress
2026-08-06 00:49:54.148 UTC [31955] WARNING: there is no transaction in progress
2026-08-06 00:49:54.164 UTC [31955] WARNING: there is no transaction in progress
2026-08-06 00:49:54.184 UTC [31955] WARNING: there is no transaction in progress
2026-08-06 00:49:54.249 UTC [31955] WARNING: there is no transaction in progress
2026-08-06 00:49:54.256 UTC [31955] WARNING: there is no transaction in progress
2026-08-06 00:49:54.272 UTC [31955] WARNING: there is no transaction in progress
2026-08-06 00:49:54.283 UTC [31955] WARNING: there is no transaction in progress
2026-08-06 00:49:54.294 UTC [31955] WARNING: there is no transaction in progress
2026-08-06 00:49:54.309 UTC [31955] WARNING: there is no transaction in progress
2026-08-06 00:49:54.318 UTC [31955] WARNING: there is no transaction in progress
2026-08-06 00:49:54.324 UTC [31955] WARNING: there is no transaction in progress
2026-08-06 00:49:54.331 UTC [31955] WARNING: there is no transaction in progress
2026-08-06 00:49:54.337 UTC [31955] WARNING: there is no transaction in progress
2026-08-06 00:49:54,953 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 00:49:54,954 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 00:49:55,193 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 00:49:55,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 00:49:55,452 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 00:49:55,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 00:49:55,712 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 00:49:55,712 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 00:49:55,974 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 00:49:56,247 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-45dx.c.gcp-upstream.internal';"
2026-08-06 00:49:56,511 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 00:49:56,511 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 00:49:56,756 [I] DB_PATCHES have already been applied
I20260806 00:49:56.762960 15950 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ranger-admin:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task3y4EW9/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 00:49:56.763022 15950 mini_ranger.cc:193] Using host: 127.15.147.148
I20260806 00:49:56.767534 15950 mini_ranger.cc:227] Ranger admin URL: http://127.15.147.148:43803
Aug 06, 2026 12:49:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:49:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:49:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 12:49:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 12:49:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 12:49:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 12:49:57 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 00:49:58.096321 15950 mini_ranger.cc:160] Time spent starting Ranger: real 9.007s user 0.000s sys 0.013s
Aug 06, 2026 12:50:18 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 12:50:18 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 12:50:18 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 12:50:19 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 12:50:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 12:50:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 12:50:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 12:50:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 12:50:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 12:50:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 12:50:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 12:50:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 12:50:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 12:50:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 12:50:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 12:50:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 12:50:19 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 12:50:19 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 00:50:20.510049 15950 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.147.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.147.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.147.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/rangerkms_127.15.147.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.147.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/rangerkms_127.15.147.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.147.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.147.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.147.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/HTTP_127.15.147.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.147.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/HTTP_127.15.147.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 00:50:20 dist-test-slave-45dx krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977420, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 00:50:20.623704 15950 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 00:50:20.635623 15950 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 00:50:21.465075 15950 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 00:50:21.465189 15950 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ranger-kms
I20260806 00:50:21.465299 15950 mini_ranger_kms.cc:227] Using postgres at 127.15.147.148:50319
2026-08-06 00:50:21,779 [I] DB FLAVOR :POSTGRES
2026-08-06 00:50:21,779 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 00:50:21,779 [I] Checking connection
2026-08-06 00:50:21,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 00:50:22,019 [I] connection success
2026-08-06 00:50:22,020 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 00:50:22,020 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 00:50:22,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 00:50:22,294 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 00:50:22,295 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 00:50:22,295 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 00:50:22,295 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:50319/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 00:50:22,564 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 00:50:23.062683 15950 mini_ranger_kms.cc:310] Created kms service
I20260806 00:50:23.274326 15950 mini_ranger_kms.cc:326] Created kudu user
I20260806 00:50:23.315479 15950 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 00:50:23 dist-test-slave-45dx krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977420, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
I20260806 00:50:23.734488 15950 mini_ranger_kms.cc:384] Added ranger policy
I20260806 00:50:23.734660 15950 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ranger-kms:/tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 00:50:23.734686 15950 mini_ranger_kms.cc:240] Using host: 127.15.147.148
I20260806 00:50:23.741055 15950 mini_ranger_kms.cc:276] Ranger KMS PID: 32219
I20260806 00:50:23.741204 15950 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.147.148:44737
Aug 06, 2026 12:50:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:50:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:50:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 12:50:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 12:50:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 12:50:25 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 00:50:25 dist-test-slave-45dx krb5kdc[31640](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785977425, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.147.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:50:26.441272 15950 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.818s user 0.000s sys 0.297s
I20260806 00:50:26.441404 15950 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 00:50:26.441507 15950 mini_ranger_kms.cc:401] 127.15.147.148:44737/kms/v1/keys
Aug 06 00:50:26 dist-test-slave-45dx krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977425, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.147.148@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
Aug 06 00:50:26 dist-test-slave-45dx krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977425, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.147.148@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
Aug 06 00:50:29 dist-test-slave-45dx krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:50:29.228749 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:44799
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:42453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:44799
--ranger_config_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.147.148:44737/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:50:29.345068 32291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:50:29.345402 32291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:50:29.345487 32291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:50:29.349418 32291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:50:29.349494 32291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:50:29.349531 32291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:50:29.349586 32291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:50:29.349622 32291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:50:29.354753 32291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:42453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.147.148:44737/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:44799
--ranger_config_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:44799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:50:29.355795 32291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:50:29.356849 32291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:50:29.365849 32303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:50:29.365860 32291 server_base.cc:1034] running on GCE node
W20260806 00:50:29.365868 32301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:50:29.366060 32305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:50:29.366616 32291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:50:29.367565 32291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:50:29.368717 32291 hybrid_clock.cc:648] HybridClock initialized: now 1785977429368694 us; error 35 us; skew 500 ppm
Aug 06 00:50:29 dist-test-slave-45dx krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977429, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:50:29.374567 32291 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:50:29.375465 32291 webserver.cc:460] Webserver started at http://127.15.147.190:33093/ using document root <none> and password file <none>
I20260806 00:50:29.376034 32291 fs_manager.cc:359] Metadata directory not provided
I20260806 00:50:29.376087 32291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:50:29.376293 32291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:50:29 dist-test-slave-45dx krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977429, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
I20260806 00:50:29.432644 32291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "a6e8fcbe7a0c482fa5292440243eb7cb"
format_stamp: "Formatted at 2026-08-06 00:50:29 on dist-test-slave-45dx"
server_key: "724716f06050a50041272c2bf6d517c3"
server_key_iv: "46e996cb33ed3d25ef4255d893a24a54"
server_key_version: "kuduclusterkey@0"
I20260806 00:50:29.433293 32291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "a6e8fcbe7a0c482fa5292440243eb7cb"
format_stamp: "Formatted at 2026-08-06 00:50:29 on dist-test-slave-45dx"
server_key: "724716f06050a50041272c2bf6d517c3"
server_key_iv: "46e996cb33ed3d25ef4255d893a24a54"
server_key_version: "kuduclusterkey@0"
I20260806 00:50:29.437528 32291 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 00:50:29.451360 32316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:50:29.452847 32291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:50:29.453003 32291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "a6e8fcbe7a0c482fa5292440243eb7cb"
format_stamp: "Formatted at 2026-08-06 00:50:29 on dist-test-slave-45dx"
server_key: "724716f06050a50041272c2bf6d517c3"
server_key_iv: "46e996cb33ed3d25ef4255d893a24a54"
server_key_version: "kuduclusterkey@0"
I20260806 00:50:29.453132 32291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:50:29.481453 32291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:50:29.484743 32291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:50:29.484961 32291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:50:29.493269 32291 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:44799
I20260806 00:50:29.493299 32392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:44799 every 8 connection(s)
I20260806 00:50:29.494313 32291 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:50:29.497069 32395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:50:29.497781 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 32291
I20260806 00:50:29.497916 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:50:29.498291 15950 external_mini_cluster.cc:1428] Setting key 586d3cda4a7a8f2a6b0d0601dcff3de9
I20260806 00:50:29.503777 32395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb: Bootstrap starting.
I20260806 00:50:29.506270 32395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb: Neither blocks nor log segments found. Creating new log.
I20260806 00:50:29.507014 32395 log.cc:824] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:50:29 dist-test-slave-45dx krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:50:29.509251 32395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb: No bootstrap required, opened a new log
I20260806 00:50:29.513492 32395 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e8fcbe7a0c482fa5292440243eb7cb" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44799 } }
I20260806 00:50:29.513764 32395 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:50:29.513823 32395 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6e8fcbe7a0c482fa5292440243eb7cb, State: Initialized, Role: FOLLOWER
I20260806 00:50:29.514014 32399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:29.499714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45596 (local address 127.15.147.190:44799)
0806 00:50:29.500121 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0806 00:50:29.500140 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:50:29.500183 (+ 43us) server_negotiation.cc:407] Connection header received
0806 00:50:29.500985 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:50:29.501009 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:50:29.501417 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:50:29.501819 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:50:29.502463 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:29.503910 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:29.504817 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:29.505496 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:29.505642 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:29.508329 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:50:29.508335 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:50:29.508350 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:50:29.508377 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:50:29.510982 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:50:29.511394 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:50:29.511403 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:50:29.511409 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:50:29.511556 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:50:29.511819 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:50:29.511822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:50:29.511824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:50:29.512372 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:50:29.512565 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 00:50:29.512807 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 00:50:29.513083 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":115,"threads_started":1}
I20260806 00:50:29.514302 32395 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e8fcbe7a0c482fa5292440243eb7cb" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44799 } }
I20260806 00:50:29.514469 32395 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:50:29.514551 32395 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:50:29.514647 32395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:50:29.515540 32395 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e8fcbe7a0c482fa5292440243eb7cb" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44799 } }
I20260806 00:50:29.515878 32395 leader_election.cc:302] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6e8fcbe7a0c482fa5292440243eb7cb; no voters:
I20260806 00:50:29.516219 32395 leader_election.cc:288] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:50:29.516345 32401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:50:29.516628 32401 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb [term 1 LEADER]: Becoming Leader. State: Replica: a6e8fcbe7a0c482fa5292440243eb7cb, State: Running, Role: LEADER
I20260806 00:50:29.516935 32401 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e8fcbe7a0c482fa5292440243eb7cb" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44799 } }
I20260806 00:50:29.517572 32395 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:50:29.518719 32404 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6e8fcbe7a0c482fa5292440243eb7cb. Latest consensus state: current_term: 1 leader_uuid: "a6e8fcbe7a0c482fa5292440243eb7cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e8fcbe7a0c482fa5292440243eb7cb" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44799 } } }
I20260806 00:50:29.518843 32403 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6e8fcbe7a0c482fa5292440243eb7cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e8fcbe7a0c482fa5292440243eb7cb" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44799 } } }
I20260806 00:50:29.518877 32404 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb [sys.catalog]: This master's current role is: LEADER
I20260806 00:50:29.518982 32403 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb [sys.catalog]: This master's current role is: LEADER
I20260806 00:50:29.519924 32418 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:50:29.520538 32395 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 00:50:29.526491 32418 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:50:29.533468 32418 catalog_manager.cc:1347] Generated new cluster ID: 11a0a2a17864407399cc3ea372f0c0df
I20260806 00:50:29.533568 32418 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:50:29.551178 32418 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:50:29.552491 32418 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:50:29.571506 32418 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb: Generated new TSK 0
I20260806 00:50:29.572402 32418 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
I20260806 00:50:30.231629 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.147.148:44737/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:44799
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:42453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:50:30.397627 32468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:50:30.397928 32468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:50:30.398023 32468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:50:30.402885 32468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:50:30.402971 32468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:50:30.403079 32468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:50:30.409965 32468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:42453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.147.148:44737/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:44799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:50:30.411348 32468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:50:30.412659 32468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:50:30.424434 32478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:50:30.424453 32479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:50:30.424453 32482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:50:30.424907 32468 server_base.cc:1034] running on GCE node
I20260806 00:50:30.425532 32468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:50:30.426395 32468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:50:30.427598 32468 hybrid_clock.cc:648] HybridClock initialized: now 1785977430427570 us; error 42 us; skew 500 ppm
Aug 06 00:50:30 dist-test-slave-45dx krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977430, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:50:30.431943 32468 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:50:30.433097 32468 webserver.cc:460] Webserver started at http://127.15.147.129:37889/ using document root <none> and password file <none>
I20260806 00:50:30.433655 32468 fs_manager.cc:359] Metadata directory not provided
I20260806 00:50:30.433717 32468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:50:30.433899 32468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:50:30 dist-test-slave-45dx krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977430, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
I20260806 00:50:30.455308 32468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "809c6522d8c742a69355e5b44cd9ce32"
format_stamp: "Formatted at 2026-08-06 00:50:30 on dist-test-slave-45dx"
server_key: "e34260fa7700f9e4b395ab6769934446"
server_key_iv: "7f1a45819d63e420b9334b14499e568a"
server_key_version: "kuduclusterkey@0"
I20260806 00:50:30.456108 32468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "809c6522d8c742a69355e5b44cd9ce32"
format_stamp: "Formatted at 2026-08-06 00:50:30 on dist-test-slave-45dx"
server_key: "e34260fa7700f9e4b395ab6769934446"
server_key_iv: "7f1a45819d63e420b9334b14499e568a"
server_key_version: "kuduclusterkey@0"
I20260806 00:50:30.461688 32468 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 00:50:30.483238 32492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:50:30.485205 32468 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.001s
I20260806 00:50:30.485375 32468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "809c6522d8c742a69355e5b44cd9ce32"
format_stamp: "Formatted at 2026-08-06 00:50:30 on dist-test-slave-45dx"
server_key: "e34260fa7700f9e4b395ab6769934446"
server_key_iv: "7f1a45819d63e420b9334b14499e568a"
server_key_version: "kuduclusterkey@0"
I20260806 00:50:30.485550 32468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:50:30.518555 32468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:50:30.522505 32468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:50:30.522734 32468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:50:30.523435 32468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:50:30.524637 32468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:50:30.524700 32468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:50:30.524761 32468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:50:30.524787 32468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:50:30.536727 32468 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:43087
I20260806 00:50:30.538048 32468 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:50:30.541147 32656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:43087 every 8 connection(s)
I20260806 00:50:30.545125 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 32468
I20260806 00:50:30.545284 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:50:30.545635 15950 external_mini_cluster.cc:1428] Setting key c9684ad05d2ad3ce99bf814d43b96e6c
WARNING: no policy specified for kudu/127.15.147.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.130@KRBTEST.COM" created.
Aug 06 00:50:30 dist-test-slave-45dx krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977430, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:50:30.574867 32663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:30.541528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:44799 (local address 127.15.147.129:50783)
0806 00:50:30.543443 (+ 1915us) negotiation.cc:107] Waiting for socket to connect
0806 00:50:30.543503 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 00:50:30.544626 (+ 1123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:50:30.545953 (+ 1327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:50:30.545963 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:50:30.546911 (+ 948us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:50:30.547625 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:50:30.547645 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:50:30.562819 (+ 15174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:50:30.562823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:50:30.564567 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:50:30.564586 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:50:30.565356 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:50:30.565376 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:50:30.565607 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:30.566394 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:50:30.566418 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:50:30.568770 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:50:30.571215 (+ 2445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:50:30.571221 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:50:30.571242 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:50:30.571622 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:50:30.571993 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:50:30.571998 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:50:30.572000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:50:30.572193 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:50:30.572739 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:50:30.572746 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:50:30.573094 (+ 348us) client_negotiation.cc:769] Sending connection context
0806 00:50:30.573318 (+ 224us) client_negotiation.cc:240] Negotiation successful
0806 00:50:30.573707 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1655,"thread_start_us":160,"threads_started":1}
I20260806 00:50:30.575981 32661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:30.540564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:50783 (local address 127.15.147.190:44799)
0806 00:50:30.545118 (+ 4554us) server_negotiation.cc:206] Beginning negotiation
0806 00:50:30.545129 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:50:30.545146 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:50:30.545284 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:50:30.545288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:50:30.545425 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:50:30.545542 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:50:30.547824 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:30.562539 (+ 14715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:30.564746 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:30.565129 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:30.565253 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:30.568997 (+ 3744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:50:30.569001 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:50:30.569004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:50:30.569037 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:50:30.571068 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:50:30.571760 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:50:30.571764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:50:30.571765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:50:30.571873 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:50:30.572351 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:50:30.572354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:50:30.572356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:50:30.572602 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:50:30.572715 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 00:50:30.575658 (+ 2943us) server_negotiation.cc:299] Negotiation successful
0806 00:50:30.575805 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4461,"thread_start_us":126,"threads_started":1}
I20260806 00:50:30.577636 32658 heartbeater.cc:347] Connected to a master server at 127.15.147.190:44799
I20260806 00:50:30.577894 32658 heartbeater.cc:464] Registering TS with master...
I20260806 00:50:30.578500 32658 heartbeater.cc:511] Master 127.15.147.190:44799 requested a full tablet report, sending...
I20260806 00:50:30.580498 32342 ts_manager.cc:194] Registered new tserver with Master: 809c6522d8c742a69355e5b44cd9ce32 (127.15.147.129:43087)
I20260806 00:50:30.582125 32342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:50783
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
I20260806 00:50:30.591784 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.147.148:44737/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:44799
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:42453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:50:30 dist-test-slave-45dx krb5kdc[31640](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785977430, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 00:50:30.761636 32666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:50:30.762074 32666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:50:30.762178 32666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:50:30.768465 32666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:50:30.768549 32666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:50:30.768651 32666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:50:30.777050 32666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:42453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.147.148:44737/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:44799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:50:30.778659 32666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:50:30.780151 32666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:50:30.794435 32676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:50:30.794812 32679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:50:30.795092 32677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:50:30.795426 32666 server_base.cc:1034] running on GCE node
I20260806 00:50:30.795943 32666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:50:30.796847 32666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:50:30.800153 32666 hybrid_clock.cc:648] HybridClock initialized: now 1785977430800122 us; error 35 us; skew 500 ppm
Aug 06 00:50:30 dist-test-slave-45dx krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977430, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:50:30.804205 32666 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:50:30.805457 32666 webserver.cc:460] Webserver started at http://127.15.147.130:34275/ using document root <none> and password file <none>
I20260806 00:50:30.806228 32666 fs_manager.cc:359] Metadata directory not provided
I20260806 00:50:30.806329 32666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:50:30.806587 32666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:50:30 dist-test-slave-45dx krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977430, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
I20260806 00:50:30.835003 32666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "544944dd91894786971d063c4bbfd28a"
format_stamp: "Formatted at 2026-08-06 00:50:30 on dist-test-slave-45dx"
server_key: "ac7edff97e0a55272ac60c2a1cb0625d"
server_key_iv: "d7b7628b63726b25d20dc81e99ab72f6"
server_key_version: "kuduclusterkey@0"
I20260806 00:50:30.835922 32666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "544944dd91894786971d063c4bbfd28a"
format_stamp: "Formatted at 2026-08-06 00:50:30 on dist-test-slave-45dx"
server_key: "ac7edff97e0a55272ac60c2a1cb0625d"
server_key_iv: "d7b7628b63726b25d20dc81e99ab72f6"
server_key_version: "kuduclusterkey@0"
I20260806 00:50:30.842710 32666 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 00:50:30.869735 32691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:50:30.871326 32666 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260806 00:50:30.871470 32666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "544944dd91894786971d063c4bbfd28a"
format_stamp: "Formatted at 2026-08-06 00:50:30 on dist-test-slave-45dx"
server_key: "ac7edff97e0a55272ac60c2a1cb0625d"
server_key_iv: "d7b7628b63726b25d20dc81e99ab72f6"
server_key_version: "kuduclusterkey@0"
I20260806 00:50:30.871623 32666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:50:30.885489 32666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:50:30.889145 32666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:50:30.889374 32666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:50:30.890123 32666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:50:30.891240 32666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:50:30.891312 32666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:50:30.891387 32666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:50:30.891427 32666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:50:30.904413 32666 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:35445
I20260806 00:50:30.905720 32666 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:50:30.915138 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 32666
I20260806 00:50:30.915293 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:50:30.915580 15950 external_mini_cluster.cc:1428] Setting key 8654f5d354207f0d00ec2600369a4877
I20260806 00:50:30.917667 393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:35445 every 8 connection(s)
WARNING: no policy specified for kudu/127.15.147.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.131@KRBTEST.COM" created.
Aug 06 00:50:30 dist-test-slave-45dx krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977430, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:50:30.944543 32661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:30.927048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:56307 (local address 127.15.147.190:44799)
0806 00:50:30.927174 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 00:50:30.927179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:50:30.929525 (+ 2346us) server_negotiation.cc:407] Connection header received
0806 00:50:30.930821 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:50:30.930825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:50:30.930883 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:50:30.930987 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:50:30.932436 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:30.933235 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:30.934893 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:30.935235 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:30.935302 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:30.940107 (+ 4805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:50:30.940112 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:50:30.940115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:50:30.940162 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:50:30.941996 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:50:30.942657 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:50:30.942661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:50:30.942662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:50:30.942754 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:50:30.943169 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:50:30.943173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:50:30.943174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:50:30.943392 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:50:30.943489 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 00:50:30.944194 (+ 705us) server_negotiation.cc:299] Negotiation successful
0806 00:50:30.944368 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:50:30.946146 397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:30.925814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:44799 (local address 127.15.147.130:56307)
0806 00:50:30.929324 (+ 3510us) negotiation.cc:107] Waiting for socket to connect
0806 00:50:30.929386 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 00:50:30.930546 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:50:30.931197 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:50:30.931206 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:50:30.931548 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:50:30.932221 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:50:30.932256 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:50:30.933401 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:50:30.933404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:50:30.934760 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:50:30.934771 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:50:30.935498 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:50:30.935501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:50:30.935692 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:30.938170 (+ 2478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:50:30.938193 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:50:30.939887 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:50:30.942129 (+ 2242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:50:30.942135 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:50:30.942151 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:50:30.942536 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:50:30.942860 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:50:30.942863 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:50:30.942865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:50:30.943058 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:50:30.943558 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:50:30.943564 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:50:30.943865 (+ 301us) client_negotiation.cc:769] Sending connection context
0806 00:50:30.944751 (+ 886us) client_negotiation.cc:240] Negotiation successful
0806 00:50:30.945103 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3275,"thread_start_us":173,"threads_started":1}
I20260806 00:50:30.947537 394 heartbeater.cc:347] Connected to a master server at 127.15.147.190:44799
I20260806 00:50:30.947815 394 heartbeater.cc:464] Registering TS with master...
I20260806 00:50:30.948603 394 heartbeater.cc:511] Master 127.15.147.190:44799 requested a full tablet report, sending...
I20260806 00:50:30.949882 32342 ts_manager.cc:194] Registered new tserver with Master: 544944dd91894786971d063c4bbfd28a (127.15.147.130:35445)
I20260806 00:50:30.950605 32342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:56307
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
I20260806 00:50:30.956133 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.147.148:44737/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:44799
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:42453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:50:31.119961 400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:50:31.120390 400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:50:31.120466 400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:50:31.126083 400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:50:31.126153 400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:50:31.126235 400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:50:31.133724 400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:42453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.147.148:44737/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:44799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:50:31.135098 400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:50:31.136442 400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:50:31.149503 408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:50:31.149503 406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:50:31.149513 410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:50:31.150542 400 server_base.cc:1034] running on GCE node
I20260806 00:50:31.151001 400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:50:31.151750 400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:50:31.153121 400 hybrid_clock.cc:648] HybridClock initialized: now 1785977431153052 us; error 82 us; skew 500 ppm
Aug 06 00:50:31 dist-test-slave-45dx krb5kdc[31640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977431, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:50:31.156966 400 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:50:31.158286 400 webserver.cc:460] Webserver started at http://127.15.147.131:41433/ using document root <none> and password file <none>
I20260806 00:50:31.159062 400 fs_manager.cc:359] Metadata directory not provided
I20260806 00:50:31.159201 400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:50:31.159503 400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:50:31 dist-test-slave-45dx krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977431, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
Aug 06 00:50:31 dist-test-slave-45dx krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977430, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
I20260806 00:50:31.180256 400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "0c506e1838cb4d85bcc64ebf22f7454b"
format_stamp: "Formatted at 2026-08-06 00:50:31 on dist-test-slave-45dx"
server_key: "603797906bca763627d357067794333e"
server_key_iv: "16cff9627bcea01a9b88891b924e1425"
server_key_version: "kuduclusterkey@0"
I20260806 00:50:31.192067 400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "0c506e1838cb4d85bcc64ebf22f7454b"
format_stamp: "Formatted at 2026-08-06 00:50:31 on dist-test-slave-45dx"
server_key: "603797906bca763627d357067794333e"
server_key_iv: "16cff9627bcea01a9b88891b924e1425"
server_key_version: "kuduclusterkey@0"
I20260806 00:50:31.211269 400 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.002s sys 0.002s
I20260806 00:50:31.234120 421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:50:31.236239 400 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.001s
I20260806 00:50:31.236462 400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "0c506e1838cb4d85bcc64ebf22f7454b"
format_stamp: "Formatted at 2026-08-06 00:50:31 on dist-test-slave-45dx"
server_key: "603797906bca763627d357067794333e"
server_key_iv: "16cff9627bcea01a9b88891b924e1425"
server_key_version: "kuduclusterkey@0"
I20260806 00:50:31.236636 400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:50:31.264947 400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:50:31.279567 400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:50:31.279803 400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:50:31.280558 400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:50:31.281706 400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:50:31.281796 400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:50:31.281869 400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:50:31.281900 400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:50:31.305455 400 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:46105
I20260806 00:50:31.306681 400 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:50:31.313714 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 400
I20260806 00:50:31.313711 582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:46105 every 8 connection(s)
I20260806 00:50:31.313849 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:50:31.314136 15950 external_mini_cluster.cc:1428] Setting key 4a1dbdba41e05c1c0df97d2c5dbe1914
Aug 06 00:50:31 dist-test-slave-45dx krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977431, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:50:31.351717 32661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:31.322513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:34297 (local address 127.15.147.190:44799)
0806 00:50:31.322631 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 00:50:31.322635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:50:31.337023 (+ 14388us) server_negotiation.cc:407] Connection header received
0806 00:50:31.338440 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:50:31.338444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:50:31.338504 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:50:31.338556 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:50:31.340868 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.341691 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.343461 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.343803 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.343902 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:31.347109 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:50:31.347113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:50:31.347115 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:50:31.347142 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:50:31.349001 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:50:31.349717 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:50:31.349721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:50:31.349723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:50:31.349825 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:50:31.350328 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:50:31.350332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:50:31.350333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:50:31.350576 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:50:31.350670 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 00:50:31.351402 (+ 732us) server_negotiation.cc:299] Negotiation successful
0806 00:50:31.351545 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:50:31.353515 592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:31.321384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:44799 (local address 127.15.147.131:34297)
0806 00:50:31.336792 (+ 15408us) negotiation.cc:107] Waiting for socket to connect
0806 00:50:31.336888 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0806 00:50:31.338142 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:50:31.339471 (+ 1329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:50:31.339482 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:50:31.339885 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:50:31.340657 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:50:31.340678 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.341884 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.341887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:50:31.343292 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:50:31.343305 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.344124 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.344128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:50:31.344367 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:31.345108 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:50:31.345130 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:50:31.346885 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:50:31.349161 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:50:31.349167 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:50:31.349185 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:50:31.349584 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:50:31.349949 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:50:31.349952 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:50:31.349954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:50:31.350192 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:50:31.350746 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:50:31.350753 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:50:31.351087 (+ 334us) client_negotiation.cc:769] Sending connection context
0806 00:50:31.351960 (+ 873us) client_negotiation.cc:240] Negotiation successful
0806 00:50:31.352374 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15067,"thread_start_us":163,"threads_started":1}
I20260806 00:50:31.354821 589 heartbeater.cc:347] Connected to a master server at 127.15.147.190:44799
I20260806 00:50:31.355075 589 heartbeater.cc:464] Registering TS with master...
I20260806 00:50:31.355670 589 heartbeater.cc:511] Master 127.15.147.190:44799 requested a full tablet report, sending...
I20260806 00:50:31.356947 32342 ts_manager.cc:194] Registered new tserver with Master: 0c506e1838cb4d85bcc64ebf22f7454b (127.15.147.131:46105)
I20260806 00:50:31.357676 32342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:34297
I20260806 00:50:31.368556 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:50:31.393213 32661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:31.376746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45610 (local address 127.15.147.190:44799)
0806 00:50:31.376896 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 00:50:31.376900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:50:31.383712 (+ 6812us) server_negotiation.cc:407] Connection header received
0806 00:50:31.383924 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:50:31.383927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:50:31.383980 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:50:31.384031 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:50:31.385070 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.385905 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.387112 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.387438 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.387514 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:31.389395 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:50:31.389401 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:50:31.389406 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:50:31.389437 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:50:31.391201 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:50:31.391754 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:50:31.391758 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:50:31.391759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:50:31.391848 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:50:31.392136 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:50:31.392139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:50:31.392141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:50:31.392418 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:50:31.392538 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 00:50:31.392921 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 00:50:31.393044 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 00:50:31.398088 32342 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:50:31.402741 32342 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:50:31.435297 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:31.417461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:46105 (local address 127.0.0.1:50750)
0806 00:50:31.429478 (+ 12017us) negotiation.cc:107] Waiting for socket to connect
0806 00:50:31.429493 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:50:31.429625 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:50:31.430338 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:50:31.430341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:50:31.430359 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:50:31.430460 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:50:31.430468 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.431847 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.431850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:50:31.433327 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:50:31.433338 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.434658 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.434662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:50:31.434795 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:31.434883 (+ 88us) client_negotiation.cc:769] Sending connection context
0806 00:50:31.434986 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 00:50:31.435130 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11948,"thread_start_us":103,"threads_started":1}
I20260806 00:50:31.435925 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:31.417936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50750 (local address 127.15.147.131:46105)
0806 00:50:31.419458 (+ 1522us) server_negotiation.cc:206] Beginning negotiation
0806 00:50:31.419473 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:50:31.429734 (+ 10261us) server_negotiation.cc:407] Connection header received
0806 00:50:31.429848 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:50:31.429852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:50:31.430039 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:50:31.430183 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:50:31.430597 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.431706 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.433499 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.434329 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.434446 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:31.434601 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 00:50:31.435673 (+ 1072us) server_negotiation.cc:299] Negotiation successful
0806 00:50:31.435787 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1400,"thread_start_us":97,"threads_started":1}
I20260806 00:50:31.439610 491 tablet_service.cc:1467] Processing CreateTablet for tablet a6a9495bdfb445d2bf34bc751b3f511c (DEFAULT_TABLE table=test-table [id=17ecebe1e4e8477c9bb59ede47a5cd38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:50:31.441613 491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6a9495bdfb445d2bf34bc751b3f511c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:50:31.452972 613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:31.416780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:43087 (local address 127.0.0.1:59616)
0806 00:50:31.428776 (+ 11996us) negotiation.cc:107] Waiting for socket to connect
0806 00:50:31.428790 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:50:31.429017 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:50:31.429818 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:50:31.429821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:50:31.429838 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:50:31.429933 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:50:31.429938 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.432001 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.432004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:50:31.434462 (+ 2458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:50:31.434472 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.452506 (+ 18034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.452510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:50:31.452628 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:31.452657 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 00:50:31.452727 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 00:50:31.452807 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11933,"mutex_wait_us":1357,"thread_start_us":90,"threads_started":1}
I20260806 00:50:31.453657 614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:31.416963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59616 (local address 127.15.147.129:43087)
0806 00:50:31.421286 (+ 4323us) server_negotiation.cc:206] Beginning negotiation
0806 00:50:31.421293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:50:31.429139 (+ 7846us) server_negotiation.cc:407] Connection header received
0806 00:50:31.429238 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:50:31.429244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:50:31.429477 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:50:31.429650 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:50:31.430066 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.431890 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.434593 (+ 2703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.436129 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.436284 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:31.453327 (+ 17043us) server_negotiation.cc:1035] Waiting for connection context
0806 00:50:31.453387 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 00:50:31.453516 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4203,"thread_start_us":114,"threads_started":1}
I20260806 00:50:31.457687 32567 tablet_service.cc:1467] Processing CreateTablet for tablet a6a9495bdfb445d2bf34bc751b3f511c (DEFAULT_TABLE table=test-table [id=17ecebe1e4e8477c9bb59ede47a5cd38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:50:31.458698 611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:31.414820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:35445 (local address 127.0.0.1:60940)
0806 00:50:31.420774 (+ 5954us) negotiation.cc:107] Waiting for socket to connect
0806 00:50:31.420823 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 00:50:31.420972 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:50:31.428360 (+ 7388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:50:31.428368 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:50:31.428397 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:50:31.428556 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:50:31.428562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.454552 (+ 25990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.454555 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:50:31.456727 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:50:31.456741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.458274 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.458280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:50:31.458395 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:31.458429 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 00:50:31.458486 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 00:50:31.458562 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5754,"thread_start_us":95,"threads_started":1}
I20260806 00:50:31.458841 32567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6a9495bdfb445d2bf34bc751b3f511c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:50:31.459401 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:31.414958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60940 (local address 127.15.147.130:35445)
0806 00:50:31.427336 (+ 12378us) server_negotiation.cc:206] Beginning negotiation
0806 00:50:31.427356 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 00:50:31.427386 (+ 30us) server_negotiation.cc:407] Connection header received
0806 00:50:31.427476 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:50:31.427481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:50:31.427736 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:50:31.428112 (+ 376us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:50:31.432124 (+ 4012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.433329 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.456917 (+ 23588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.458108 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.458979 (+ 871us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:31.459124 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 00:50:31.459175 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 00:50:31.459272 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12239,"thread_start_us":112,"threads_started":1}
I20260806 00:50:31.460911 617 tablet_bootstrap.cc:492] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b: Bootstrap starting.
I20260806 00:50:31.462347 32740 tablet_service.cc:1467] Processing CreateTablet for tablet a6a9495bdfb445d2bf34bc751b3f511c (DEFAULT_TABLE table=test-table [id=17ecebe1e4e8477c9bb59ede47a5cd38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:50:31.463362 32740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6a9495bdfb445d2bf34bc751b3f511c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:50:31.463887 617 tablet_bootstrap.cc:654] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b: Neither blocks nor log segments found. Creating new log.
I20260806 00:50:31.464967 617 log.cc:824] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b: Log is configured to *not* fsync() on all Append() calls
I20260806 00:50:31.472887 619 tablet_bootstrap.cc:492] T a6a9495bdfb445d2bf34bc751b3f511c P 809c6522d8c742a69355e5b44cd9ce32: Bootstrap starting.
I20260806 00:50:31.477617 619 tablet_bootstrap.cc:654] T a6a9495bdfb445d2bf34bc751b3f511c P 809c6522d8c742a69355e5b44cd9ce32: Neither blocks nor log segments found. Creating new log.
I20260806 00:50:31.478794 619 log.cc:824] T a6a9495bdfb445d2bf34bc751b3f511c P 809c6522d8c742a69355e5b44cd9ce32: Log is configured to *not* fsync() on all Append() calls
I20260806 00:50:31.482214 617 tablet_bootstrap.cc:492] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b: No bootstrap required, opened a new log
I20260806 00:50:31.482461 617 ts_tablet_manager.cc:1397] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.006s
I20260806 00:50:31.490496 619 tablet_bootstrap.cc:492] T a6a9495bdfb445d2bf34bc751b3f511c P 809c6522d8c742a69355e5b44cd9ce32: No bootstrap required, opened a new log
I20260806 00:50:31.490718 619 ts_tablet_manager.cc:1397] T a6a9495bdfb445d2bf34bc751b3f511c P 809c6522d8c742a69355e5b44cd9ce32: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.000s
I20260806 00:50:31.492648 617 raft_consensus.cc:348] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544944dd91894786971d063c4bbfd28a" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 35445 } } peers { permanent_uuid: "809c6522d8c742a69355e5b44cd9ce32" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 43087 } } peers { permanent_uuid: "0c506e1838cb4d85bcc64ebf22f7454b" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46105 } }
I20260806 00:50:31.493060 617 raft_consensus.cc:374] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:50:31.493153 617 raft_consensus.cc:729] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c506e1838cb4d85bcc64ebf22f7454b, State: Initialized, Role: FOLLOWER
I20260806 00:50:31.493736 617 consensus_queue.cc:260] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544944dd91894786971d063c4bbfd28a" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 35445 } } peers { permanent_uuid: "809c6522d8c742a69355e5b44cd9ce32" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 43087 } } peers { permanent_uuid: "0c506e1838cb4d85bcc64ebf22f7454b" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46105 } }
I20260806 00:50:31.494918 617 ts_tablet_manager.cc:1428] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260806 00:50:31.495309 589 heartbeater.cc:503] Master 127.15.147.190:44799 was elected leader, sending a full tablet report...
I20260806 00:50:31.499845 620 tablet_bootstrap.cc:492] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a: Bootstrap starting.
I20260806 00:50:31.507268 623 raft_consensus.cc:482] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:50:31.507500 623 raft_consensus.cc:504] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544944dd91894786971d063c4bbfd28a" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 35445 } } peers { permanent_uuid: "809c6522d8c742a69355e5b44cd9ce32" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 43087 } } peers { permanent_uuid: "0c506e1838cb4d85bcc64ebf22f7454b" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46105 } }
I20260806 00:50:31.510402 620 tablet_bootstrap.cc:654] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a: Neither blocks nor log segments found. Creating new log.
I20260806 00:50:31.502112 619 raft_consensus.cc:348] T a6a9495bdfb445d2bf34bc751b3f511c P 809c6522d8c742a69355e5b44cd9ce32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544944dd91894786971d063c4bbfd28a" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 35445 } } peers { permanent_uuid: "809c6522d8c742a69355e5b44cd9ce32" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 43087 } } peers { permanent_uuid: "0c506e1838cb4d85bcc64ebf22f7454b" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46105 } }
I20260806 00:50:31.513479 619 raft_consensus.cc:374] T a6a9495bdfb445d2bf34bc751b3f511c P 809c6522d8c742a69355e5b44cd9ce32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:50:31.513543 619 raft_consensus.cc:729] T a6a9495bdfb445d2bf34bc751b3f511c P 809c6522d8c742a69355e5b44cd9ce32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 809c6522d8c742a69355e5b44cd9ce32, State: Initialized, Role: FOLLOWER
I20260806 00:50:31.514072 619 consensus_queue.cc:260] T a6a9495bdfb445d2bf34bc751b3f511c P 809c6522d8c742a69355e5b44cd9ce32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544944dd91894786971d063c4bbfd28a" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 35445 } } peers { permanent_uuid: "809c6522d8c742a69355e5b44cd9ce32" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 43087 } } peers { permanent_uuid: "0c506e1838cb4d85bcc64ebf22f7454b" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46105 } }
I20260806 00:50:31.515183 619 ts_tablet_manager.cc:1428] T a6a9495bdfb445d2bf34bc751b3f511c P 809c6522d8c742a69355e5b44cd9ce32: Time spent starting tablet: real 0.024s user 0.008s sys 0.000s
I20260806 00:50:31.515549 32658 heartbeater.cc:503] Master 127.15.147.190:44799 was elected leader, sending a full tablet report...
I20260806 00:50:31.517414 592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:31.511792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:35445 (local address 127.15.147.131:42621)
0806 00:50:31.512050 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 00:50:31.512066 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:50:31.512152 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:50:31.512785 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:50:31.512788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:50:31.512807 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:50:31.512925 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:50:31.512932 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.514716 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.514720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:50:31.515981 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:50:31.515993 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.517021 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.517025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:50:31.517128 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:31.517150 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:50:31.517216 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 00:50:31.517287 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260806 00:50:31.518056 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:31.512338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:42621 (local address 127.15.147.130:35445)
0806 00:50:31.512458 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 00:50:31.512463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:50:31.512478 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:50:31.512528 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:50:31.512531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:50:31.512576 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:50:31.512649 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:50:31.513060 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.513769 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.516163 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.516801 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.516913 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:31.516962 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 00:50:31.517868 (+ 906us) server_negotiation.cc:299] Negotiation successful
0806 00:50:31.517943 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:50:31.518872 316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6a9495bdfb445d2bf34bc751b3f511c" candidate_uuid: "0c506e1838cb4d85bcc64ebf22f7454b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "544944dd91894786971d063c4bbfd28a" is_pre_election: true
W20260806 00:50:31.525962 425 leader_election.cc:341] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 544944dd91894786971d063c4bbfd28a (127.15.147.130:35445): Illegal state: must be running to vote when last-logged opid is not known
I20260806 00:50:31.538647 592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:31.525231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:43087 (local address 127.15.147.131:40247)
0806 00:50:31.526306 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0806 00:50:31.526323 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:50:31.526455 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:50:31.526880 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:50:31.526884 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:50:31.526905 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:50:31.527003 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:50:31.527008 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.528012 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.528017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:50:31.529208 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:50:31.529223 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.538182 (+ 8959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.538186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:50:31.538295 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:31.538320 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 00:50:31.538396 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 00:50:31.538490 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":981}
I20260806 00:50:31.539180 614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:31.524975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:40247 (local address 127.15.147.129:43087)
0806 00:50:31.525105 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 00:50:31.525110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:50:31.526555 (+ 1445us) server_negotiation.cc:407] Connection header received
0806 00:50:31.526619 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:50:31.526622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:50:31.526675 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:50:31.526741 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:50:31.527134 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.527836 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.536527 (+ 8691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.537239 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.537358 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:31.537415 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 00:50:31.538969 (+ 1554us) server_negotiation.cc:299] Negotiation successful
0806 00:50:31.539061 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:50:31.539608 623 leader_election.cc:288] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 544944dd91894786971d063c4bbfd28a (127.15.147.130:35445), 809c6522d8c742a69355e5b44cd9ce32 (127.15.147.129:43087)
I20260806 00:50:31.539848 32594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6a9495bdfb445d2bf34bc751b3f511c" candidate_uuid: "0c506e1838cb4d85bcc64ebf22f7454b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "809c6522d8c742a69355e5b44cd9ce32" is_pre_election: true
I20260806 00:50:31.540233 32594 raft_consensus.cc:2454] T a6a9495bdfb445d2bf34bc751b3f511c P 809c6522d8c742a69355e5b44cd9ce32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c506e1838cb4d85bcc64ebf22f7454b in term 0.
I20260806 00:50:31.540771 423 leader_election.cc:302] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c506e1838cb4d85bcc64ebf22f7454b, 809c6522d8c742a69355e5b44cd9ce32; no voters: 544944dd91894786971d063c4bbfd28a
I20260806 00:50:31.541353 622 raft_consensus.cc:2781] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:50:31.541458 622 raft_consensus.cc:482] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:50:31.541504 622 raft_consensus.cc:3037] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:50:31.542909 622 raft_consensus.cc:504] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544944dd91894786971d063c4bbfd28a" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 35445 } } peers { permanent_uuid: "809c6522d8c742a69355e5b44cd9ce32" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 43087 } } peers { permanent_uuid: "0c506e1838cb4d85bcc64ebf22f7454b" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46105 } }
I20260806 00:50:31.543684 32594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6a9495bdfb445d2bf34bc751b3f511c" candidate_uuid: "0c506e1838cb4d85bcc64ebf22f7454b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "809c6522d8c742a69355e5b44cd9ce32"
I20260806 00:50:31.543815 32594 raft_consensus.cc:3037] T a6a9495bdfb445d2bf34bc751b3f511c P 809c6522d8c742a69355e5b44cd9ce32 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:50:31.545156 32594 raft_consensus.cc:2454] T a6a9495bdfb445d2bf34bc751b3f511c P 809c6522d8c742a69355e5b44cd9ce32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c506e1838cb4d85bcc64ebf22f7454b in term 1.
I20260806 00:50:31.545511 423 leader_election.cc:302] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c506e1838cb4d85bcc64ebf22f7454b, 809c6522d8c742a69355e5b44cd9ce32; no voters:
I20260806 00:50:31.545702 623 raft_consensus.cc:2781] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:50:31.545892 623 raft_consensus.cc:686] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [term 1 LEADER]: Becoming Leader. State: Replica: 0c506e1838cb4d85bcc64ebf22f7454b, State: Running, Role: LEADER
I20260806 00:50:31.546226 623 consensus_queue.cc:237] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544944dd91894786971d063c4bbfd28a" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 35445 } } peers { permanent_uuid: "809c6522d8c742a69355e5b44cd9ce32" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 43087 } } peers { permanent_uuid: "0c506e1838cb4d85bcc64ebf22f7454b" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46105 } }
W20260806 00:50:31.550748 32659 tablet.cc:2369] T a6a9495bdfb445d2bf34bc751b3f511c P 809c6522d8c742a69355e5b44cd9ce32: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:50:31.551519 622 leader_election.cc:288] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [CANDIDATE]: Term 1 election: Requested vote from peers 544944dd91894786971d063c4bbfd28a (127.15.147.130:35445), 809c6522d8c742a69355e5b44cd9ce32 (127.15.147.129:43087)
I20260806 00:50:31.551751 316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6a9495bdfb445d2bf34bc751b3f511c" candidate_uuid: "0c506e1838cb4d85bcc64ebf22f7454b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "544944dd91894786971d063c4bbfd28a"
W20260806 00:50:31.552199 425 leader_election.cc:341] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 544944dd91894786971d063c4bbfd28a (127.15.147.130:35445): Illegal state: must be running to vote when last-logged opid is not known
I20260806 00:50:31.553314 32342 catalog_manager.cc:5495] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c506e1838cb4d85bcc64ebf22f7454b (127.15.147.131). New cstate: current_term: 1 leader_uuid: "0c506e1838cb4d85bcc64ebf22f7454b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544944dd91894786971d063c4bbfd28a" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 35445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "809c6522d8c742a69355e5b44cd9ce32" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 43087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c506e1838cb4d85bcc64ebf22f7454b" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46105 } health_report { overall_health: HEALTHY } } }
W20260806 00:50:31.569741 590 tablet.cc:2369] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:50:31.571792 620 log.cc:824] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a: Log is configured to *not* fsync() on all Append() calls
I20260806 00:50:31.577451 620 tablet_bootstrap.cc:492] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a: No bootstrap required, opened a new log
I20260806 00:50:31.577826 620 ts_tablet_manager.cc:1397] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a: Time spent bootstrapping tablet: real 0.078s user 0.005s sys 0.002s
I20260806 00:50:31.579506 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:31.574614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50762 (local address 127.15.147.131:46105)
0806 00:50:31.574737 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:50:31.574742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:50:31.574755 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:50:31.574834 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:50:31.574837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:50:31.574910 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:50:31.575010 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:50:31.575325 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.576100 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.577338 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:50:31.577663 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:50:31.577762 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:50:31.578201 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:50:31.578703 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:50:31.578829 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 00:50:31.579267 (+ 438us) server_negotiation.cc:299] Negotiation successful
0806 00:50:31.579331 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:50:31.581523 620 raft_consensus.cc:348] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544944dd91894786971d063c4bbfd28a" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 35445 } } peers { permanent_uuid: "809c6522d8c742a69355e5b44cd9ce32" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 43087 } } peers { permanent_uuid: "0c506e1838cb4d85bcc64ebf22f7454b" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46105 } }
I20260806 00:50:31.581878 620 raft_consensus.cc:374] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:50:31.581990 620 raft_consensus.cc:729] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 544944dd91894786971d063c4bbfd28a, State: Initialized, Role: FOLLOWER
I20260806 00:50:31.582538 620 consensus_queue.cc:260] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "544944dd91894786971d063c4bbfd28a" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 35445 } } peers { permanent_uuid: "809c6522d8c742a69355e5b44cd9ce32" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 43087 } } peers { permanent_uuid: "0c506e1838cb4d85bcc64ebf22f7454b" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46105 } }
I20260806 00:50:31.583598 620 ts_tablet_manager.cc:1428] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260806 00:50:31.583992 394 heartbeater.cc:503] Master 127.15.147.190:44799 was elected leader, sending a full tablet report...
I20260806 00:50:31.590205 32594 raft_consensus.cc:1261] T a6a9495bdfb445d2bf34bc751b3f511c P 809c6522d8c742a69355e5b44cd9ce32 [term 1 FOLLOWER]: Refusing update from remote peer 0c506e1838cb4d85bcc64ebf22f7454b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:50:31.590265 316 raft_consensus.cc:3037] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:50:31.590937 625 consensus_queue.cc:1035] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [LEADER]: Connected to new peer: Peer: permanent_uuid: "809c6522d8c742a69355e5b44cd9ce32" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 43087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:50:31.591677 316 raft_consensus.cc:1261] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a [term 1 FOLLOWER]: Refusing update from remote peer 0c506e1838cb4d85bcc64ebf22f7454b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:50:31.593524 625 consensus_queue.cc:1035] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [LEADER]: Connected to new peer: Peer: permanent_uuid: "544944dd91894786971d063c4bbfd28a" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 35445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:50:31.608407 631 mvcc.cc:205] Tried to move back new op lower bound from 7315363559785435136 to 7315363559615836160. Current Snapshot: MvccSnapshot[applied={T|T < 7315363559785435136}]
I20260806 00:50:31.618848 636 mvcc.cc:205] Tried to move back new op lower bound from 7315363559785435136 to 7315363559615836160. Current Snapshot: MvccSnapshot[applied={T|T < 7315363559785435136 or (T in {7315363559785435136})}]
Aug 06 00:50:31 dist-test-slave-45dx krb5kdc[31640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977430, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
I20260806 00:50:31.628999 638 mvcc.cc:205] Tried to move back new op lower bound from 7315363559785435136 to 7315363559615836160. Current Snapshot: MvccSnapshot[applied={T|T < 7315363559785435136}]
I20260806 00:50:31.640117 32338 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:50:31.640336 32338 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:50:31.643417 32338 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6e8fcbe7a0c482fa5292440243eb7cb: Sending DeleteTablet for 3 replicas of tablet a6a9495bdfb445d2bf34bc751b3f511c
I20260806 00:50:31.644842 32567 tablet_service.cc:1514] Processing DeleteTablet for tablet a6a9495bdfb445d2bf34bc751b3f511c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:50:31 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:59616
I20260806 00:50:31.645896 32740 tablet_service.cc:1514] Processing DeleteTablet for tablet a6a9495bdfb445d2bf34bc751b3f511c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:50:31 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:60940
I20260806 00:50:31.646660 491 tablet_service.cc:1514] Processing DeleteTablet for tablet a6a9495bdfb445d2bf34bc751b3f511c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:50:31 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:50750
I20260806 00:50:31.647505 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 32468
I20260806 00:50:31.665601 647 tablet_replica.cc:330] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a: stopping tablet replica
I20260806 00:50:31.666085 647 raft_consensus.cc:2229] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:50:31.666524 647 raft_consensus.cc:2258] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:50:31.668546 649 tablet_replica.cc:330] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b: stopping tablet replica
I20260806 00:50:31.668902 649 raft_consensus.cc:2229] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:50:31.669327 649 raft_consensus.cc:2258] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:50:31.671238 647 ts_tablet_manager.cc:1905] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:50:31.673847 649 ts_tablet_manager.cc:1905] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:50:31.684860 647 ts_tablet_manager.cc:1918] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:50:31.684994 647 log.cc:1196] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/a6a9495bdfb445d2bf34bc751b3f511c
I20260806 00:50:31.685362 647 ts_tablet_manager.cc:1939] T a6a9495bdfb445d2bf34bc751b3f511c P 544944dd91894786971d063c4bbfd28a: Deleting consensus metadata
I20260806 00:50:31.686794 32319 catalog_manager.cc:4841] TS 544944dd91894786971d063c4bbfd28a (127.15.147.130:35445): tablet a6a9495bdfb445d2bf34bc751b3f511c (table test-table [id=17ecebe1e4e8477c9bb59ede47a5cd38]) successfully deleted
I20260806 00:50:31.687528 649 ts_tablet_manager.cc:1918] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:50:31.687642 649 log.cc:1196] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/a6a9495bdfb445d2bf34bc751b3f511c
I20260806 00:50:31.688007 649 ts_tablet_manager.cc:1939] T a6a9495bdfb445d2bf34bc751b3f511c P 0c506e1838cb4d85bcc64ebf22f7454b: Deleting consensus metadata
I20260806 00:50:31.689183 32320 catalog_manager.cc:4841] TS 0c506e1838cb4d85bcc64ebf22f7454b (127.15.147.131:46105): tablet a6a9495bdfb445d2bf34bc751b3f511c (table test-table [id=17ecebe1e4e8477c9bb59ede47a5cd38]) successfully deleted
W20260806 00:50:31.690979 32317 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.147.129:43087 (error 108)
I20260806 00:50:31.692745 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 32666
I20260806 00:50:31.707032 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 400
I20260806 00:50:31.710393 611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:50:31.709976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:43087 (local address 127.0.0.1:59632)
0806 00:50:31.710205 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 00:50:31.710287 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:43087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260806 00:50:31.710740 32317 catalog_manager.cc:4584] TS 809c6522d8c742a69355e5b44cd9ce32 (127.15.147.129:43087): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a6a9495bdfb445d2bf34bc751b3f511c: Network error: Client connection negotiation failed: client connection to 127.15.147.129:43087: connect: Connection refused (error 111)
I20260806 00:50:31.722580 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 32291
I20260806 00:50:31.743520 15950 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 00:50:32.097843 15950 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 00:50:32.097990 15950 mini_ranger.cc:66] Stopping Ranger...
I20260806 00:50:32.444075 15950 mini_ranger.cc:68] Stopped Ranger
2026-08-06 00:50:32.444 UTC [31682] LOG: received smart shutdown request
2026-08-06 00:50:32.448 UTC [31682] LOG: background worker "logical replication launcher" (PID 31691) exited with exit code 1
2026-08-06 00:50:32.448 UTC [31686] LOG: shutting down
2026-08-06 00:50:32.615 UTC [31682] LOG: database system is shut down
2026-08-06T00:50:32Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48712 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:50:32 dist-test-slave-45dx krb5kdc[664](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:50:32 dist-test-slave-45dx krb5kdc[664](info): set up 2 sockets
Aug 06 00:50:32 dist-test-slave-45dx krb5kdc[664](info): commencing operation
krb5kdc: starting...
W20260806 00:50:33.950394 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:50:34 dist-test-slave-45dx krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:50:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:50:34Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.147.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.147.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.147.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/rangeradmin_127.15.147.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.147.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/rangeradmin_127.15.147.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.147.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.147.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.147.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/rangerlookup_127.15.147.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.147.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/rangerlookup_127.15.147.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.147.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.147.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.147.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/HTTP_127.15.147.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.147.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/HTTP_127.15.147.148@KRBTEST.COM.keytab.
I20260806 00:50:34.143568 15950 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task3y4EW9/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/postgres -l logfile start
2026-08-06 00:50:36.647 UTC [705] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 00:50:36.647 UTC [705] LOG: listening on IPv4 address "127.15.147.148", port 58303
2026-08-06 00:50:36.651 UTC [705] LOG: listening on Unix socket "/tmp/.s.PGSQL.58303"
2026-08-06 00:50:36.690 UTC [709] LOG: database system was shut down at 2026-08-06 00:50:35 UTC
2026-08-06 00:50:36.695 UTC [705] LOG: database system is ready to accept connections
I20260806 00:50:37.877451 15950 mini_postgres.cc:96] Postgres bound to 58303
2026-08-06 00:50:37.885 UTC [724] FATAL: database "slave" does not exist
127.15.147.148:58303 - accepting connections
I20260806 00:50:37.886147 15950 mini_ranger.cc:161] Starting Ranger...
I20260806 00:50:37.902508 15950 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 00:50:38.394373 15950 mini_ranger.cc:87] Created ranger Postgres database
I20260806 00:50:38.394455 15950 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-admin
2026-08-06 00:50:38,690 [I] DB FLAVOR :POSTGRES
2026-08-06 00:50:38,691 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 00:50:38,691 [I] Checking connection..
2026-08-06 00:50:38,691 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 00:50:38,934 [I] Checking connection passed.
2026-08-06 00:50:38,934 [I] --------- Verifying version history table ---------
2026-08-06 00:50:38,934 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 00:50:39,205 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 00:50:39,206 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 00:50:39,480 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 00:50:39,480 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 00:50:39,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 00:50:39.736 UTC [810] WARNING: there is no transaction in progress
2026-08-06 00:50:39,745 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 00:50:39,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 00:50:40,002 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 00:50:40,002 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 00:50:40,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 00:50:40,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 00:50:40,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-45dx.c.gcp-upstream.internal','N') ;"
2026-08-06 00:50:40,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 00:50:41,095 [I] Table x_portal_user does not exist in database ranger
2026-08-06 00:50:41,095 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 00:50:41,368 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 00:50:41,369 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 00:50:41,633 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 00:50:41,894 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 00:50:41,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 00:50:42.869 UTC [953] WARNING: there is no transaction in progress
2026-08-06 00:50:42.886 UTC [953] WARNING: there is no transaction in progress
2026-08-06 00:50:42.902 UTC [953] WARNING: there is no transaction in progress
2026-08-06 00:50:42.916 UTC [953] WARNING: there is no transaction in progress
2026-08-06 00:50:42.936 UTC [953] WARNING: there is no transaction in progress
2026-08-06 00:50:42.958 UTC [953] WARNING: there is no transaction in progress
2026-08-06 00:50:43.015 UTC [953] WARNING: there is no transaction in progress
2026-08-06 00:50:43.022 UTC [953] WARNING: there is no transaction in progress
2026-08-06 00:50:43.037 UTC [953] WARNING: there is no transaction in progress
2026-08-06 00:50:43.047 UTC [953] WARNING: there is no transaction in progress
2026-08-06 00:50:43.059 UTC [953] WARNING: there is no transaction in progress
2026-08-06 00:50:43.073 UTC [953] WARNING: there is no transaction in progress
2026-08-06 00:50:43.082 UTC [953] WARNING: there is no transaction in progress
2026-08-06 00:50:43.088 UTC [953] WARNING: there is no transaction in progress
2026-08-06 00:50:43.094 UTC [953] WARNING: there is no transaction in progress
2026-08-06 00:50:43.101 UTC [953] WARNING: there is no transaction in progress
2026-08-06 00:50:43,725 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 00:50:43,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 00:50:43,972 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 00:50:43,972 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 00:50:44,239 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 00:50:44,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 00:50:44,504 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 00:50:44,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 00:50:44,761 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 00:50:45,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-45dx.c.gcp-upstream.internal';"
2026-08-06 00:50:45,275 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 00:50:45,275 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 00:50:45,535 [I] DB_PATCHES have already been applied
I20260806 00:50:45.542059 15950 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-admin:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task3y4EW9/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 00:50:45.542130 15950 mini_ranger.cc:193] Using host: 127.15.147.148
I20260806 00:50:45.546766 15950 mini_ranger.cc:227] Ranger admin URL: http://127.15.147.148:43093
Aug 06, 2026 12:50:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:50:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:50:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 12:50:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 12:50:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 12:50:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 12:50:46 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 00:50:47.208046 15950 mini_ranger.cc:160] Time spent starting Ranger: real 9.322s user 0.000s sys 0.012s
Aug 06, 2026 12:51:07 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 12:51:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 12:51:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 12:51:07 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 12:51:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 12:51:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 12:51:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 12:51:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 12:51:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 12:51:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 12:51:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 12:51:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 12:51:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 12:51:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 12:51:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 12:51:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 12:51:07 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 12:51:08 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 00:51:09.019711 15950 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.147.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.147.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.147.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/rangerkms_127.15.147.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.147.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/rangerkms_127.15.147.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.147.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.147.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.147.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/HTTP_127.15.147.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.147.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/HTTP_127.15.147.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 00:51:09 dist-test-slave-45dx krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977469, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 00:51:09.114920 15950 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 00:51:09.126787 15950 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 00:51:10.004828 15950 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 00:51:10.004913 15950 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-kms
I20260806 00:51:10.004928 15950 mini_ranger_kms.cc:227] Using postgres at 127.15.147.148:58303
2026-08-06 00:51:10,196 [I] DB FLAVOR :POSTGRES
2026-08-06 00:51:10,196 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 00:51:10,196 [I] Checking connection
2026-08-06 00:51:10,196 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 00:51:10,447 [I] connection success
2026-08-06 00:51:10,447 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 00:51:10,447 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 00:51:10,447 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 00:51:10,709 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 00:51:10,709 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 00:51:10,709 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 00:51:10,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.147.148:58303/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 00:51:10,984 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 00:51:11.446298 15950 mini_ranger_kms.cc:310] Created kms service
I20260806 00:51:11.646111 15950 mini_ranger_kms.cc:326] Created kudu user
I20260806 00:51:11.695981 15950 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 00:51:11 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977469, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
I20260806 00:51:12.126543 15950 mini_ranger_kms.cc:384] Added ranger policy
I20260806 00:51:12.126717 15950 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-kms:/tmp/dist-test-task3y4EW9/build/debug/bin/postgresql.jar:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task3y4EW9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 00:51:12.126740 15950 mini_ranger_kms.cc:240] Using host: 127.15.147.148
I20260806 00:51:12.131773 15950 mini_ranger_kms.cc:276] Ranger KMS PID: 1216
I20260806 00:51:12.131868 15950 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.147.148:33351
Aug 06, 2026 12:51:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 12:51:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 12:51:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 12:51:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 12:51:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 12:51:13 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 00:51:14 dist-test-slave-45dx krb5kdc[664](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785977474, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.147.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:14.827778 15950 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.713s user 0.010s sys 0.168s
I20260806 00:51:14.827936 15950 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 00:51:14.828007 15950 mini_ranger_kms.cc:401] 127.15.147.148:33351/kms/v1/keys
Aug 06 00:51:14 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977474, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.147.148@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
Aug 06 00:51:15 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977474, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.147.148@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
Aug 06 00:51:17 dist-test-slave-45dx krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:51:17.571862 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:43551
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:43551
--ranger_config_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.147.148:33351/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:51:17.685727 1291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:17.685998 1291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:17.686038 1291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:17.690097 1291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:51:17.690151 1291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:17.690169 1291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:17.690184 1291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:51:17.690199 1291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:51:17.695463 1291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.147.148:33351/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:43551
--ranger_config_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:43551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:17.696543 1291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:17.697561 1291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:17.706276 1301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:17.706286 1296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:17.706321 1291 server_base.cc:1034] running on GCE node
W20260806 00:51:17.706286 1299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:17.707177 1291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:17.708261 1291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:17.709451 1291 hybrid_clock.cc:648] HybridClock initialized: now 1785977477709426 us; error 38 us; skew 500 ppm
Aug 06 00:51:17 dist-test-slave-45dx krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977477, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:17.715513 1291 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:51:17.716427 1291 webserver.cc:460] Webserver started at http://127.15.147.190:43385/ using document root <none> and password file <none>
I20260806 00:51:17.716931 1291 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:17.716992 1291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:17.717165 1291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:51:17 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977477, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
I20260806 00:51:17.779433 1291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "2b6d7603a60e4119ac9a0db6a09058db"
format_stamp: "Formatted at 2026-08-06 00:51:17 on dist-test-slave-45dx"
server_key: "fb76f54f719b5fc9bf58652bde2bd7ee"
server_key_iv: "21afdd851b9ba7459d8679f9d00c5f7d"
server_key_version: "kuduclusterkey@0"
I20260806 00:51:17.780113 1291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "2b6d7603a60e4119ac9a0db6a09058db"
format_stamp: "Formatted at 2026-08-06 00:51:17 on dist-test-slave-45dx"
server_key: "fb76f54f719b5fc9bf58652bde2bd7ee"
server_key_iv: "21afdd851b9ba7459d8679f9d00c5f7d"
server_key_version: "kuduclusterkey@0"
I20260806 00:51:17.784138 1291 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260806 00:51:17.802708 1308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:17.804136 1291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:51:17.804287 1291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "2b6d7603a60e4119ac9a0db6a09058db"
format_stamp: "Formatted at 2026-08-06 00:51:17 on dist-test-slave-45dx"
server_key: "fb76f54f719b5fc9bf58652bde2bd7ee"
server_key_iv: "21afdd851b9ba7459d8679f9d00c5f7d"
server_key_version: "kuduclusterkey@0"
I20260806 00:51:17.804452 1291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:17.833964 1291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:17.836824 1291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:17.837049 1291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:17.844717 1291 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:43551
I20260806 00:51:17.844724 1379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:43551 every 8 connection(s)
I20260806 00:51:17.845711 1291 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:51:17.848673 1380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:17.850544 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 1291
I20260806 00:51:17.850673 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:51:17.850981 15950 external_mini_cluster.cc:1428] Setting key d15cdf655bb175e395724f01f401fdc4
I20260806 00:51:17.855363 1380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db: Bootstrap starting.
I20260806 00:51:17.858035 1380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:17.859015 1380 log.cc:824] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:51:17 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:51:17.861477 1380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db: No bootstrap required, opened a new log
I20260806 00:51:17.865275 1380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6d7603a60e4119ac9a0db6a09058db" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 43551 } }
I20260806 00:51:17.865543 1380 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:51:17.865621 1380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6d7603a60e4119ac9a0db6a09058db, State: Initialized, Role: FOLLOWER
I20260806 00:51:17.866043 1380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6d7603a60e4119ac9a0db6a09058db" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 43551 } }
I20260806 00:51:17.866163 1380 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:51:17.866248 1380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:51:17.866367 1385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:17.852437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35648 (local address 127.15.147.190:43551)
0806 00:51:17.852825 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:17.852842 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:17.852878 (+ 36us) server_negotiation.cc:407] Connection header received
0806 00:51:17.853500 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:17.853516 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:17.853842 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:17.854110 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:17.855077 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:17.856093 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:17.857290 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:17.857749 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:17.857869 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:17.860995 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:17.861001 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:17.861014 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:17.861039 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:17.863277 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:17.863788 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:17.863793 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:17.863798 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:17.863916 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:17.864212 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:17.864214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:17.864215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:17.864668 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:17.864834 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:17.865128 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 00:51:17.865432 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":118,"threads_started":1}
I20260806 00:51:17.866384 1380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:17.867390 1380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6d7603a60e4119ac9a0db6a09058db" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 43551 } }
I20260806 00:51:17.867739 1380 leader_election.cc:302] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b6d7603a60e4119ac9a0db6a09058db; no voters:
I20260806 00:51:17.868070 1380 leader_election.cc:288] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:51:17.868191 1388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:51:17.868431 1388 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [term 1 LEADER]: Becoming Leader. State: Replica: 2b6d7603a60e4119ac9a0db6a09058db, State: Running, Role: LEADER
I20260806 00:51:17.869016 1388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6d7603a60e4119ac9a0db6a09058db" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 43551 } }
I20260806 00:51:17.869346 1380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:51:17.870635 1389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b6d7603a60e4119ac9a0db6a09058db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6d7603a60e4119ac9a0db6a09058db" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 43551 } } }
I20260806 00:51:17.870764 1389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [sys.catalog]: This master's current role is: LEADER
I20260806 00:51:17.871125 1392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b6d7603a60e4119ac9a0db6a09058db. Latest consensus state: current_term: 1 leader_uuid: "2b6d7603a60e4119ac9a0db6a09058db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6d7603a60e4119ac9a0db6a09058db" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 43551 } } }
I20260806 00:51:17.871212 1403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:51:17.871212 1392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [sys.catalog]: This master's current role is: LEADER
I20260806 00:51:17.871688 1380 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 00:51:17.877803 1403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:51:17.885113 1403 catalog_manager.cc:1347] Generated new cluster ID: 6b49ec4733f54a8bb00fd5356d9fe208
I20260806 00:51:17.885183 1403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:51:17.909138 1403 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:51:17.910193 1403 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:51:17.928673 1403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db: Generated new TSK 0
I20260806 00:51:17.929418 1403 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
I20260806 00:51:18.604496 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.147.148:33351/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:43551
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:51:18.744316 1452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:18.744719 1452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:18.744810 1452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:18.749814 1452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:18.749877 1452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:18.749965 1452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:51:18.755403 1452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.147.148:33351/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:43551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:18.756678 1452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:18.757747 1452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:18.769241 1461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:18.769511 1452 server_base.cc:1034] running on GCE node
W20260806 00:51:18.769256 1462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:18.769757 1464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:18.770202 1452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:18.771082 1452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:18.772275 1452 hybrid_clock.cc:648] HybridClock initialized: now 1785977478772175 us; error 113 us; skew 500 ppm
Aug 06 00:51:18 dist-test-slave-45dx krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977478, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:18.775980 1452 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:51:18.777212 1452 webserver.cc:460] Webserver started at http://127.15.147.129:40705/ using document root <none> and password file <none>
I20260806 00:51:18.778013 1452 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:18.778164 1452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:18.778457 1452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:51:18 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977478, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
I20260806 00:51:18.797202 1452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "85611f732dad42a19e452e4d73568174"
format_stamp: "Formatted at 2026-08-06 00:51:18 on dist-test-slave-45dx"
server_key: "1377e9f91fcbc74e18b9eef56d26742e"
server_key_iv: "606b2b9fa3a707b2183f02846baee7dd"
server_key_version: "kuduclusterkey@0"
I20260806 00:51:18.797967 1452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "85611f732dad42a19e452e4d73568174"
format_stamp: "Formatted at 2026-08-06 00:51:18 on dist-test-slave-45dx"
server_key: "1377e9f91fcbc74e18b9eef56d26742e"
server_key_iv: "606b2b9fa3a707b2183f02846baee7dd"
server_key_version: "kuduclusterkey@0"
I20260806 00:51:18.803089 1452 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 00:51:18.825614 1471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:18.827301 1452 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.003s sys 0.000s
I20260806 00:51:18.827436 1452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "85611f732dad42a19e452e4d73568174"
format_stamp: "Formatted at 2026-08-06 00:51:18 on dist-test-slave-45dx"
server_key: "1377e9f91fcbc74e18b9eef56d26742e"
server_key_iv: "606b2b9fa3a707b2183f02846baee7dd"
server_key_version: "kuduclusterkey@0"
I20260806 00:51:18.827581 1452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:18.851620 1452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:18.855283 1452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:18.855536 1452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:18.856176 1452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:51:18.857344 1452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:51:18.857470 1452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:18.857555 1452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:51:18.857631 1452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:18.868994 1452 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:38655
I20260806 00:51:18.870347 1452 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:51:18.871532 1631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:38655 every 8 connection(s)
I20260806 00:51:18.883824 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 1452
I20260806 00:51:18.883945 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
Aug 06 00:51:18 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977478, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:51:18.884274 15950 external_mini_cluster.cc:1428] Setting key 395dc3d335e1ed643293c4df470c5e04
I20260806 00:51:18.893539 1634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:18.873319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:47829 (local address 127.15.147.190:43551)
0806 00:51:18.873633 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:18.873642 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:18.876847 (+ 3205us) server_negotiation.cc:407] Connection header received
0806 00:51:18.876900 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:18.876903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:18.876961 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:18.877048 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:18.878530 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:18.879419 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:18.881177 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:18.881538 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:18.881635 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:18.888682 (+ 7047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:18.888686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:18.888689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:18.888717 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:18.890841 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:18.891551 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:18.891555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:18.891557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:18.891658 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:18.892101 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:18.892107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:18.892108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:18.892383 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:18.892471 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:18.893183 (+ 712us) server_negotiation.cc:299] Negotiation successful
0806 00:51:18.893337 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":130,"threads_started":1}
I20260806 00:51:18.895579 1636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:18.873798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:43551 (local address 127.15.147.129:47829)
0806 00:51:18.875454 (+ 1656us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:18.875507 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:18.876563 (+ 1056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:18.877360 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:18.877369 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:18.877711 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:18.878359 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:18.878377 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:18.879573 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:18.879579 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:18.881033 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:18.881045 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:18.881843 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:18.881847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:18.882039 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:18.882801 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:18.882825 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:18.888475 (+ 5650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:18.891004 (+ 2529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:18.891011 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:18.891031 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:18.891426 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:18.891784 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:18.891786 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:18.891788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:18.891983 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:18.892541 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:18.892548 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:18.892880 (+ 332us) client_negotiation.cc:769] Sending connection context
0806 00:51:18.894051 (+ 1171us) client_negotiation.cc:240] Negotiation successful
0806 00:51:18.894449 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1438,"thread_start_us":126,"threads_started":1}
I20260806 00:51:18.897243 1632 heartbeater.cc:347] Connected to a master server at 127.15.147.190:43551
I20260806 00:51:18.897516 1632 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.147.130@KRBTEST.COM; defaulting to no policy
I20260806 00:51:18.898115 1632 heartbeater.cc:511] Master 127.15.147.190:43551 requested a full tablet report, sending...
I20260806 00:51:18.900292 1330 ts_manager.cc:194] Registered new tserver with Master: 85611f732dad42a19e452e4d73568174 (127.15.147.129:38655)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.130@KRBTEST.COM" created.
I20260806 00:51:18.902066 1330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:47829
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
I20260806 00:51:18.923483 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.147.148:33351/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:43551
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:51:18 dist-test-slave-45dx krb5kdc[664](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785977478, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 00:51:19.066988 1641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:19.067269 1641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:19.067406 1641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:19.071753 1641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:19.071828 1641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:19.071919 1641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:51:19.077200 1641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.147.148:33351/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:43551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:19.078452 1641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:19.079491 1641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:19.091058 1650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:19.091087 1653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:19.091493 1651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:19.091698 1641 server_base.cc:1034] running on GCE node
I20260806 00:51:19.092132 1641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:19.092914 1641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:19.094170 1641 hybrid_clock.cc:648] HybridClock initialized: now 1785977479094146 us; error 43 us; skew 500 ppm
Aug 06 00:51:19 dist-test-slave-45dx krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977479, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:19.098002 1641 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:51:19.099251 1641 webserver.cc:460] Webserver started at http://127.15.147.130:36003/ using document root <none> and password file <none>
I20260806 00:51:19.100041 1641 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:19.100160 1641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:19.100531 1641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:51:19 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977479, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
I20260806 00:51:19.123598 1641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9"
format_stamp: "Formatted at 2026-08-06 00:51:19 on dist-test-slave-45dx"
server_key: "14b601e9781ec0924dd58f4595ff7b9e"
server_key_iv: "e4f866f82912301dfd88fbc3fa1ae732"
server_key_version: "kuduclusterkey@0"
I20260806 00:51:19.124681 1641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9"
format_stamp: "Formatted at 2026-08-06 00:51:19 on dist-test-slave-45dx"
server_key: "14b601e9781ec0924dd58f4595ff7b9e"
server_key_iv: "e4f866f82912301dfd88fbc3fa1ae732"
server_key_version: "kuduclusterkey@0"
I20260806 00:51:19.130728 1641 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 00:51:19.152674 1664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:19.154366 1641 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260806 00:51:19.154496 1641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9"
format_stamp: "Formatted at 2026-08-06 00:51:19 on dist-test-slave-45dx"
server_key: "14b601e9781ec0924dd58f4595ff7b9e"
server_key_iv: "e4f866f82912301dfd88fbc3fa1ae732"
server_key_version: "kuduclusterkey@0"
I20260806 00:51:19.154623 1641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:19.181973 1641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:19.185376 1641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:19.185590 1641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:19.186271 1641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:51:19.187193 1641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:51:19.187264 1641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:19.187351 1641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:51:19.187394 1641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:19.199981 1641 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:33505
I20260806 00:51:19.199990 1811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:33505 every 8 connection(s)
I20260806 00:51:19.201232 1641 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:51:19.204370 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 1641
I20260806 00:51:19.204495 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:51:19.204818 15950 external_mini_cluster.cc:1428] Setting key 3e9c2bc35234eab867ffa56fbfd551b4
Aug 06 00:51:19 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977479, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.147.131@KRBTEST.COM; defaulting to no policy
I20260806 00:51:19.221230 1634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:19.203654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:56195 (local address 127.15.147.190:43551)
0806 00:51:19.203820 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:19.203824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:19.207090 (+ 3266us) server_negotiation.cc:407] Connection header received
0806 00:51:19.208564 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:19.208570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:19.208634 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:19.208691 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:19.210418 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:19.211198 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:19.212614 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:19.213002 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:19.213118 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:19.216605 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:19.216611 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:19.216615 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:19.216651 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:19.218626 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:19.219303 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:19.219307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:19.219311 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:19.219402 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:19.219842 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:19.219846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:19.219848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:19.220131 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:19.220235 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:19.220899 (+ 664us) server_negotiation.cc:299] Negotiation successful
0806 00:51:19.221032 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 00:51:19.222335 1816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:19.204168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:43551 (local address 127.15.147.130:56195)
0806 00:51:19.206927 (+ 2759us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:19.206993 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:19.208275 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:19.209137 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:19.209149 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:19.209556 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:19.210237 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:19.210257 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:19.211379 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:19.211382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:19.212493 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:19.212503 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:19.213302 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:19.213308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:19.213528 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:19.214264 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:19.214291 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:19.216371 (+ 2080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:19.218774 (+ 2403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:19.218783 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:19.218814 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:19.219175 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:19.219530 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:19.219535 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:19.219540 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:19.219723 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:19.220306 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:19.220312 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:19.220639 (+ 327us) client_negotiation.cc:769] Sending connection context
0806 00:51:19.220845 (+ 206us) client_negotiation.cc:240] Negotiation successful
0806 00:51:19.221152 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2534,"thread_start_us":154,"threads_started":1}
I20260806 00:51:19.223551 1813 heartbeater.cc:347] Connected to a master server at 127.15.147.190:43551
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.131@KRBTEST.COM" created.
I20260806 00:51:19.223800 1813 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:19.224424 1813 heartbeater.cc:511] Master 127.15.147.190:43551 requested a full tablet report, sending...
I20260806 00:51:19.226248 1330 ts_manager.cc:194] Registered new tserver with Master: aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 (127.15.147.130:33505)
I20260806 00:51:19.227059 1330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:56195
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
I20260806 00:51:19.243122 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.147.148:33351/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:43551
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:51:19.421792 1820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:19.422152 1820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:19.422214 1820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:19.428398 1820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:19.428540 1820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:19.428668 1820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:51:19.437250 1820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.147.148:33351/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:43551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:19.438753 1820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:19.440344 1820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:19.458238 1827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:19.458328 1829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:19.459286 1826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:19.459896 1820 server_base.cc:1034] running on GCE node
I20260806 00:51:19.460460 1820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:19.461319 1820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Aug 06 00:51:19 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977478, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
I20260806 00:51:19.464856 1820 hybrid_clock.cc:648] HybridClock initialized: now 1785977479464802 us; error 50 us; skew 500 ppm
Aug 06 00:51:19 dist-test-slave-45dx krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977479, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:19.473685 1820 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:51:19.474972 1820 webserver.cc:460] Webserver started at http://127.15.147.131:39593/ using document root <none> and password file <none>
I20260806 00:51:19.475806 1820 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:19.475987 1820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:19.476378 1820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 00:51:19 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977479, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
I20260806 00:51:19.499456 1820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "31f99b9db8a7499cbab22bf1e3b33f83"
format_stamp: "Formatted at 2026-08-06 00:51:19 on dist-test-slave-45dx"
server_key: "5db9f5470a65b2c51e3e0e68749c6fdd"
server_key_iv: "35463df105aea210f249c7f21082ed94"
server_key_version: "kuduclusterkey@0"
I20260806 00:51:19.500650 1820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "31f99b9db8a7499cbab22bf1e3b33f83"
format_stamp: "Formatted at 2026-08-06 00:51:19 on dist-test-slave-45dx"
server_key: "5db9f5470a65b2c51e3e0e68749c6fdd"
server_key_iv: "35463df105aea210f249c7f21082ed94"
server_key_version: "kuduclusterkey@0"
I20260806 00:51:19.510196 1820 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.000s
I20260806 00:51:19.544663 1839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:19.546332 1820 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.001s
I20260806 00:51:19.546532 1820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "31f99b9db8a7499cbab22bf1e3b33f83"
format_stamp: "Formatted at 2026-08-06 00:51:19 on dist-test-slave-45dx"
server_key: "5db9f5470a65b2c51e3e0e68749c6fdd"
server_key_iv: "35463df105aea210f249c7f21082ed94"
server_key_version: "kuduclusterkey@0"
I20260806 00:51:19.546761 1820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:19.567581 1820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:19.571667 1820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:19.571870 1820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:19.572554 1820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:51:19.573554 1820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:51:19.573613 1820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:19.573664 1820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:51:19.573735 1820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260806 00:51:19.584507 1820 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:46449
I20260806 00:51:19.584601 2001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:46449 every 8 connection(s)
I20260806 00:51:19.585738 1820 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:51:19.595403 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 1820
I20260806 00:51:19.595553 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:51:19.595913 15950 external_mini_cluster.cc:1428] Setting key 7793df6d204f98ef341424425eb645f7
Aug 06 00:51:19 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977479, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:51:19.601971 1634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:19.587932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:36063 (local address 127.15.147.190:43551)
0806 00:51:19.588118 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:19.588122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:19.589278 (+ 1156us) server_negotiation.cc:407] Connection header received
0806 00:51:19.590546 (+ 1268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:19.590548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:19.590593 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:19.590651 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:19.592254 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:19.592871 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:19.594678 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:19.594889 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:19.594971 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:19.598227 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:19.598230 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:19.598233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:19.598263 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:19.599589 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:19.600337 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:19.600343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:19.600345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:19.600434 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:19.600907 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:19.600909 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:19.600910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:19.601069 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:19.601142 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:19.601753 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 00:51:19.601858 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 00:51:19.603266 2006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:19.588420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:43551 (local address 127.15.147.131:36063)
0806 00:51:19.589060 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:19.589132 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:19.590293 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:19.590894 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:19.590905 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:19.591293 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:19.592093 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:19.592116 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:19.593011 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:19.593015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:19.594541 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:19.594554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:19.595059 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:19.595063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:19.595295 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:19.596051 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:19.596072 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:19.598027 (+ 1955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:19.599713 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:19.599719 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:19.599735 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:19.600194 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:19.600539 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:19.600542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:19.600544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:19.600805 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:19.601184 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:19.601190 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:19.601501 (+ 311us) client_negotiation.cc:769] Sending connection context
0806 00:51:19.601721 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 00:51:19.602167 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":182,"threads_started":1}
I20260806 00:51:19.604496 2003 heartbeater.cc:347] Connected to a master server at 127.15.147.190:43551
I20260806 00:51:19.604804 2003 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:19.605499 2003 heartbeater.cc:511] Master 127.15.147.190:43551 requested a full tablet report, sending...
I20260806 00:51:19.606869 1330 ts_manager.cc:194] Registered new tserver with Master: 31f99b9db8a7499cbab22bf1e3b33f83 (127.15.147.131:46449)
I20260806 00:51:19.607556 1330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:36063
I20260806 00:51:19.609769 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:51:19.610525 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 1452
I20260806 00:51:19.648612 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 1641
I20260806 00:51:19.664778 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 1820
I20260806 00:51:19.681718 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 1291
I20260806 00:51:19.695091 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:43551
--webserver_interface=127.15.147.190
--webserver_port=43385
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:43551
--ranger_config_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.147.148:33351/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.147.148:33351/kms with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:51:19.803937 2007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:19.804160 2007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:19.804194 2007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:19.808082 2007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:51:19.808143 2007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:19.808163 2007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:19.808178 2007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:51:19.808238 2007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:51:19.813436 2007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.147.148:33351/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:43551
--ranger_config_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:43551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=43385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:19.814493 2007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:19.815526 2007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:19.824209 2016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:19.824226 2015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:19.824223 2021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:19.824317 2007 server_base.cc:1034] running on GCE node
I20260806 00:51:19.825093 2007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:19.826074 2007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:19.827236 2007 hybrid_clock.cc:648] HybridClock initialized: now 1785977479827210 us; error 38 us; skew 500 ppm
Aug 06 00:51:19 dist-test-slave-45dx krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977479, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:19.830008 2007 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:51:19.830893 2007 webserver.cc:460] Webserver started at http://127.15.147.190:43385/ using document root <none> and password file <none>
I20260806 00:51:19.831429 2007 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:19.831535 2007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 00:51:19 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977479, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
I20260806 00:51:19.911648 2007 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:51:19.914151 2037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:19.915551 2007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:51:19.915704 2007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "2b6d7603a60e4119ac9a0db6a09058db"
format_stamp: "Formatted at 2026-08-06 00:51:17 on dist-test-slave-45dx"
server_key: "fb76f54f719b5fc9bf58652bde2bd7ee"
server_key_iv: "21afdd851b9ba7459d8679f9d00c5f7d"
server_key_version: "kuduclusterkey@0"
I20260806 00:51:19.916289 2007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:19.926158 2007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:19.928692 2007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:19.928879 2007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:19.936031 2007 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:43551
I20260806 00:51:19.936029 2106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:43551 every 8 connection(s)
I20260806 00:51:19.937108 2007 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:51:19.941170 2108 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:51:19.942788 2108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db: Bootstrap starting.
I20260806 00:51:19.942956 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 2007
I20260806 00:51:19.943426 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.147.148:33351/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:38655
--local_ip_for_outbound_sockets=127.15.147.129
--tserver_master_addrs=127.15.147.190:43551
--webserver_port=40705
--webserver_interface=127.15.147.129
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.147.148:33351/kms with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:51:19.959455 2108 log.cc:824] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:19.966663 2108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:51:19.967098 2108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db: Bootstrap complete.
I20260806 00:51:19.971312 2108 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6d7603a60e4119ac9a0db6a09058db" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 43551 } }
I20260806 00:51:19.971745 2108 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6d7603a60e4119ac9a0db6a09058db, State: Initialized, Role: FOLLOWER
I20260806 00:51:19.972286 2108 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6d7603a60e4119ac9a0db6a09058db" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 43551 } }
I20260806 00:51:19.972488 2108 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:51:19.972558 2108 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:51:19.972651 2108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [term 1 FOLLOWER]: Advancing to term 2
I20260806 00:51:19.974545 2108 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6d7603a60e4119ac9a0db6a09058db" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 43551 } }
I20260806 00:51:19.974908 2108 leader_election.cc:302] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b6d7603a60e4119ac9a0db6a09058db; no voters:
I20260806 00:51:19.975222 2108 leader_election.cc:288] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 00:51:19.975389 2114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [term 2 FOLLOWER]: Leader election won for term 2
I20260806 00:51:19.975674 2114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [term 2 LEADER]: Becoming Leader. State: Replica: 2b6d7603a60e4119ac9a0db6a09058db, State: Running, Role: LEADER
I20260806 00:51:19.976049 2114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6d7603a60e4119ac9a0db6a09058db" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 43551 } }
I20260806 00:51:19.976565 2108 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:51:19.977525 2117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2b6d7603a60e4119ac9a0db6a09058db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6d7603a60e4119ac9a0db6a09058db" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 43551 } } }
I20260806 00:51:19.977654 2117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [sys.catalog]: This master's current role is: LEADER
I20260806 00:51:19.978444 2118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b6d7603a60e4119ac9a0db6a09058db. Latest consensus state: current_term: 2 leader_uuid: "2b6d7603a60e4119ac9a0db6a09058db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6d7603a60e4119ac9a0db6a09058db" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 43551 } } }
I20260806 00:51:19.978566 2126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:51:19.978957 2108 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 00:51:19.978624 2118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db [sys.catalog]: This master's current role is: LEADER
I20260806 00:51:19.984880 2126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:51:19.985680 2126 catalog_manager.cc:1259] Loaded cluster ID: 6b49ec4733f54a8bb00fd5356d9fe208
I20260806 00:51:19.985776 2126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:51:19.988394 2126 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:51:19.989084 2126 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db: Loaded TSK: 0
I20260806 00:51:19.990348 2126 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:51:20.097527 2110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:20.098001 2110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:20.098137 2110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:20.104604 2110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:20.104678 2110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:20.104765 2110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:51:20.112715 2110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.147.148:33351/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:38655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=40705
--tserver_master_addrs=127.15.147.190:43551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:20.114197 2110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:20.115384 2110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:20.129159 2160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:20.129590 2163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:20.129652 2161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:20.130136 2110 server_base.cc:1034] running on GCE node
I20260806 00:51:20.130611 2110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:20.131345 2110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:20.132513 2110 hybrid_clock.cc:648] HybridClock initialized: now 1785977480132501 us; error 35 us; skew 500 ppm
Aug 06 00:51:20 dist-test-slave-45dx krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977480, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:20.135373 2110 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:51:20.136426 2110 webserver.cc:460] Webserver started at http://127.15.147.129:40705/ using document root <none> and password file <none>
I20260806 00:51:20.137166 2110 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:20.137368 2110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 00:51:20 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977480, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
I20260806 00:51:20.155740 2110 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:51:20.160061 2172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:20.161589 2110 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 00:51:20.161851 2110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "85611f732dad42a19e452e4d73568174"
format_stamp: "Formatted at 2026-08-06 00:51:18 on dist-test-slave-45dx"
server_key: "1377e9f91fcbc74e18b9eef56d26742e"
server_key_iv: "606b2b9fa3a707b2183f02846baee7dd"
server_key_version: "kuduclusterkey@0"
I20260806 00:51:20.162659 2110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:20.209614 2110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:20.213665 2110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:20.213938 2110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:20.214754 2110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:51:20.215919 2110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:51:20.216043 2110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:20.216185 2110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:51:20.216284 2110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:20.228997 2110 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:38655
I20260806 00:51:20.230096 2110 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:51:20.231094 2342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:38655 every 8 connection(s)
I20260806 00:51:20.233633 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 2110
I20260806 00:51:20.234153 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.147.148:33351/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:33505
--local_ip_for_outbound_sockets=127.15.147.130
--tserver_master_addrs=127.15.147.190:43551
--webserver_port=36003
--webserver_interface=127.15.147.130
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.147.148:33351/kms with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:51:20 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977480, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:51:20.254948 2352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:20.236448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:43551 (local address 127.15.147.129:44813)
0806 00:51:20.237710 (+ 1262us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:20.237753 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:20.238764 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:20.240112 (+ 1348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:20.240134 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:20.240548 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:20.241265 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:20.241290 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:20.243469 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:20.243475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:20.245001 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:20.245015 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:20.245782 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:20.245787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:20.246014 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:20.246787 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:20.246817 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:20.248767 (+ 1950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:20.251381 (+ 2614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:20.251387 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:20.251401 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:20.251754 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:20.252181 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:20.252184 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:20.252186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:20.252380 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:20.253090 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:20.253096 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:20.253384 (+ 288us) client_negotiation.cc:769] Sending connection context
0806 00:51:20.253582 (+ 198us) client_negotiation.cc:240] Negotiation successful
0806 00:51:20.253891 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1049,"thread_start_us":173,"threads_started":1}
I20260806 00:51:20.256775 2345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:20.232929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:44813 (local address 127.15.147.190:43551)
0806 00:51:20.233416 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:20.233441 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:20.237908 (+ 4467us) server_negotiation.cc:407] Connection header received
0806 00:51:20.239246 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:20.239262 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:20.239643 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:20.239993 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:20.241509 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:20.243278 (+ 1769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:20.245155 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:20.245604 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:20.245727 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:20.249015 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:20.249022 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:20.249037 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:20.249069 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:20.251257 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:20.251905 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:20.251911 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:20.251917 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:20.252075 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:20.252509 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:20.252513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:20.252515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:20.252955 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:20.253089 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:20.255538 (+ 2449us) server_negotiation.cc:299] Negotiation successful
0806 00:51:20.255847 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":129,"threads_started":1}
I20260806 00:51:20.260535 2343 heartbeater.cc:347] Connected to a master server at 127.15.147.190:43551
I20260806 00:51:20.260784 2343 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:20.261385 2343 heartbeater.cc:511] Master 127.15.147.190:43551 requested a full tablet report, sending...
W20260806 00:51:20.263803 2343 heartbeater.cc:644] Failed to heartbeat to 127.15.147.190:43551 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 00:51:20.263978 2343 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:20.264155 2343 heartbeater.cc:511] Master 127.15.147.190:43551 requested a full tablet report, sending...
I20260806 00:51:20.265172 2343 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:20.265344 2343 heartbeater.cc:511] Master 127.15.147.190:43551 requested a full tablet report, sending...
W20260806 00:51:20.267481 2343 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:51:20.379264 2349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:20.379534 2349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:20.379578 2349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:20.383808 2349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:20.383873 2349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:20.383932 2349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:51:20.389592 2349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.147.148:33351/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:33505
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=36003
--tserver_master_addrs=127.15.147.190:43551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:20.390714 2349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:20.391774 2349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:20.401911 2357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:20.401986 2360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:20.401907 2363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:20.402312 2349 server_base.cc:1034] running on GCE node
I20260806 00:51:20.402758 2349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:20.403527 2349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:20.404734 2349 hybrid_clock.cc:648] HybridClock initialized: now 1785977480404693 us; error 52 us; skew 500 ppm
Aug 06 00:51:20 dist-test-slave-45dx krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977480, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:20.408529 2349 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:51:20.409653 2349 webserver.cc:460] Webserver started at http://127.15.147.130:36003/ using document root <none> and password file <none>
I20260806 00:51:20.410386 2349 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:20.410485 2349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 00:51:20 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977480, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
I20260806 00:51:20.447126 2349 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:51:20.450551 2376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:20.455955 2349 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260806 00:51:20.456141 2349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9"
format_stamp: "Formatted at 2026-08-06 00:51:19 on dist-test-slave-45dx"
server_key: "14b601e9781ec0924dd58f4595ff7b9e"
server_key_iv: "e4f866f82912301dfd88fbc3fa1ae732"
server_key_version: "kuduclusterkey@0"
I20260806 00:51:20.456986 2349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:20.504366 2349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:20.507658 2349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:20.507910 2349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:20.508678 2349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:51:20.509639 2349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:51:20.509711 2349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:20.509778 2349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:51:20.509835 2349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:20.521625 2349 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:33505
I20260806 00:51:20.521654 2535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:33505 every 8 connection(s)
I20260806 00:51:20.522797 2349 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:51:20.530920 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 2349
I20260806 00:51:20.531402 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.147.148:33351/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:46449
--local_ip_for_outbound_sockets=127.15.147.131
--tserver_master_addrs=127.15.147.190:43551
--webserver_port=39593
--webserver_interface=127.15.147.131
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.147.148:33351/kms with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:51:20 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977480, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:51:20.540930 2345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:20.525371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:47083 (local address 127.15.147.190:43551)
0806 00:51:20.525557 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:20.525563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:20.526217 (+ 654us) server_negotiation.cc:407] Connection header received
0806 00:51:20.527535 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:20.527539 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:20.527600 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:20.527659 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:20.529352 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:20.530047 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:20.531858 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:20.532246 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:20.532370 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:20.535815 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:20.535822 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:20.535827 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:20.535864 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:20.538074 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:20.538791 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:20.538796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:20.538800 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:20.538920 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:20.539397 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:20.539403 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:20.539407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:20.539693 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:20.539824 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:20.540504 (+ 680us) server_negotiation.cc:299] Negotiation successful
0806 00:51:20.540675 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:51:20.542398 2539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:20.525427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:43551 (local address 127.15.147.130:47083)
0806 00:51:20.526046 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:20.526106 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:20.527259 (+ 1153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:20.528084 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:20.528092 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:20.528475 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:20.529128 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:20.529149 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:20.530190 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:20.530194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:20.531697 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:20.531712 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:20.532513 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:20.532521 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:20.532720 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:20.533653 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:20.533676 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:20.535596 (+ 1920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:20.538228 (+ 2632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:20.538235 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:20.538252 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:20.538658 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:20.539049 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:20.539052 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:20.539054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:20.539272 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:20.539832 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:20.539839 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:20.540173 (+ 334us) client_negotiation.cc:769] Sending connection context
0806 00:51:20.540424 (+ 251us) client_negotiation.cc:240] Negotiation successful
0806 00:51:20.540794 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":157,"threads_started":1}
I20260806 00:51:20.543843 2536 heartbeater.cc:347] Connected to a master server at 127.15.147.190:43551
I20260806 00:51:20.544148 2536 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:20.544828 2536 heartbeater.cc:511] Master 127.15.147.190:43551 requested a full tablet report, sending...
W20260806 00:51:20.546658 2536 heartbeater.cc:644] Failed to heartbeat to 127.15.147.190:43551 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 00:51:20.546780 2536 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:20.546936 2536 heartbeater.cc:511] Master 127.15.147.190:43551 requested a full tablet report, sending...
I20260806 00:51:20.547748 2536 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:20.547899 2536 heartbeater.cc:511] Master 127.15.147.190:43551 requested a full tablet report, sending...
W20260806 00:51:20.548653 2536 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:51:20.671614 2540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:20.671901 2540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:20.672008 2540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:20.676369 2540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:20.676433 2540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:20.676518 2540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:51:20.683665 2540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.147.148:33351/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:46449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=39593
--tserver_master_addrs=127.15.147.190:43551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:20.684978 2540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:20.686024 2540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:20.696940 2553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:20.697045 2556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:20.697716 2550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:20.698544 2540 server_base.cc:1034] running on GCE node
I20260806 00:51:20.698987 2540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:20.699786 2540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:20.700994 2540 hybrid_clock.cc:648] HybridClock initialized: now 1785977480700968 us; error 41 us; skew 500 ppm
Aug 06 00:51:20 dist-test-slave-45dx krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977480, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:20.704236 2540 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:51:20.705322 2540 webserver.cc:460] Webserver started at http://127.15.147.131:39593/ using document root <none> and password file <none>
I20260806 00:51:20.705904 2540 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:20.706012 2540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 00:51:20 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977480, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
I20260806 00:51:20.726609 2540 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:51:20.729852 2568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:20.731269 2540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:51:20.731420 2540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "31f99b9db8a7499cbab22bf1e3b33f83"
format_stamp: "Formatted at 2026-08-06 00:51:19 on dist-test-slave-45dx"
server_key: "5db9f5470a65b2c51e3e0e68749c6fdd"
server_key_iv: "35463df105aea210f249c7f21082ed94"
server_key_version: "kuduclusterkey@0"
I20260806 00:51:20.731990 2540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:20.752974 2540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:20.756270 2540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:20.756577 2540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:20.757398 2540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:51:20.758541 2540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:51:20.758607 2540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:20.758663 2540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:51:20.758694 2540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:20.770360 2540 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:46449
I20260806 00:51:20.771489 2540 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:51:20.772475 2752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:46449 every 8 connection(s)
Aug 06 00:51:20 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977480, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:51:20.781672 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 2540
I20260806 00:51:20.786073 2345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:20.773569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:55491 (local address 127.15.147.190:43551)
0806 00:51:20.773701 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:20.773705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:20.774714 (+ 1009us) server_negotiation.cc:407] Connection header received
0806 00:51:20.775680 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:20.775683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:20.775739 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:20.775826 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:20.777013 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:20.777617 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:20.778852 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:20.779095 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:20.779212 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:20.781864 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:20.781868 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:20.781870 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:20.781901 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:20.783611 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:20.784174 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:20.784179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:20.784181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:20.784276 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:20.784753 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:20.784756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:20.784757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:20.784999 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:20.785116 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:20.785793 (+ 677us) server_negotiation.cc:299] Negotiation successful
0806 00:51:20.785915 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:51:20.787104 2755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:20.774017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:43551 (local address 127.15.147.131:55491)
0806 00:51:20.774510 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:20.774561 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:20.775475 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:20.775991 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:20.775998 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:20.776255 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:20.776864 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:20.776879 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:20.777783 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:20.777787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:20.778725 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:20.778735 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:20.779256 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:20.779261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:20.779437 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:20.780091 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:20.780107 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:20.781621 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:20.783744 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:20.783750 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:20.783764 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:20.784069 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:20.784407 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:20.784411 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:20.784413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:20.784633 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:20.785140 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:20.785148 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:20.785462 (+ 314us) client_negotiation.cc:769] Sending connection context
0806 00:51:20.785736 (+ 274us) client_negotiation.cc:240] Negotiation successful
0806 00:51:20.786073 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":143,"threads_started":1}
I20260806 00:51:20.788414 2753 heartbeater.cc:347] Connected to a master server at 127.15.147.190:43551
I20260806 00:51:20.788653 2753 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:20.789242 2753 heartbeater.cc:511] Master 127.15.147.190:43551 requested a full tablet report, sending...
I20260806 00:51:20.790793 2057 ts_manager.cc:194] Registered new tserver with Master: 31f99b9db8a7499cbab22bf1e3b33f83 (127.15.147.131:46449)
I20260806 00:51:20.792197 2057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:55491
I20260806 00:51:20.793430 2759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:20.783425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35650 (local address 127.15.147.190:43551)
0806 00:51:20.784304 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:20.784362 (+ 58us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:20.784376 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:51:20.784416 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:20.784418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:20.784453 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:20.784549 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:20.785470 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:20.786216 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:20.787765 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:20.788081 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:20.788188 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:20.789920 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:20.789924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:20.789926 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:20.789951 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:20.791598 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:20.792066 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:20.792070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:20.792071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:20.792164 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:20.792469 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:20.792472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:20.792474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:20.792664 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:20.792847 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:20.793084 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 00:51:20.793201 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":805,"thread_start_us":122,"threads_started":1}
Aug 06 00:51:21 dist-test-slave-45dx krb5kdc[664](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785977481, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:21.267709 2343 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:21.268051 2343 heartbeater.cc:511] Master 127.15.147.190:43551 requested a full tablet report, sending...
I20260806 00:51:21.268880 2057 ts_manager.cc:194] Registered new tserver with Master: 85611f732dad42a19e452e4d73568174 (127.15.147.129:38655)
I20260806 00:51:21.269709 2057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:44813
Aug 06 00:51:21 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977481, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
I20260806 00:51:21.548889 2536 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:21.549175 2536 heartbeater.cc:511] Master 127.15.147.190:43551 requested a full tablet report, sending...
I20260806 00:51:21.549948 2057 ts_manager.cc:194] Registered new tserver with Master: aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 (127.15.147.130:33505)
I20260806 00:51:21.550705 2057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:47083
I20260806 00:51:21.554947 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:51:21.568377 2773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:21.556490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35652 (local address 127.15.147.190:43551)
0806 00:51:21.559528 (+ 3038us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:21.559532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:21.559545 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:51:21.559580 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:21.559582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:21.559627 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:21.559738 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:21.560633 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.561429 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.562504 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.562850 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.562960 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:21.564585 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:21.564588 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:21.564591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:21.564616 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:21.566430 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:21.566995 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:21.566998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:21.567000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:21.567092 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:21.567399 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:21.567402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:21.567404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:21.567608 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:21.567836 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:21.568056 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 00:51:21.568180 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2965,"thread_start_us":65,"threads_started":1}
I20260806 00:51:21.572429 2057 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35652:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:51:21.576936 2057 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:51:21.596364 2781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:21.587815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:38655 (local address 127.0.0.1:47648)
0806 00:51:21.590223 (+ 2408us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:21.590258 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:21.590414 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:21.591116 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:21.591121 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:21.591140 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:21.591262 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:21.591269 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.593175 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.593178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:21.594468 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:21.594479 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.595779 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.595783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:21.595917 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:21.595970 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 00:51:21.596075 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 00:51:21.596186 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2272,"mutex_wait_us":24,"thread_start_us":100,"threads_started":1}
I20260806 00:51:21.597019 2783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:21.588700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47648 (local address 127.15.147.129:38655)
0806 00:51:21.588984 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:21.588991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:21.590353 (+ 1362us) server_negotiation.cc:407] Connection header received
0806 00:51:21.590573 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:21.590580 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:21.590787 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:21.590945 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:21.591517 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.593038 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.594621 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.595450 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.595564 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:21.595705 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:21.596753 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0806 00:51:21.596876 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":102,"threads_started":1}
I20260806 00:51:21.597257 2785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:21.588279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:33505 (local address 127.0.0.1:56316)
0806 00:51:21.590653 (+ 2374us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:21.590665 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:21.590741 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:21.591959 (+ 1218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:21.591962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:21.591978 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:21.592086 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:21.592092 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.593890 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.593894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:21.595036 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:21.595046 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.596882 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.596885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:21.596981 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:21.597004 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:51:21.597056 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 00:51:21.597130 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2318,"mutex_wait_us":1,"thread_start_us":120,"threads_started":1}
I20260806 00:51:21.597966 2784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:21.589059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56316 (local address 127.15.147.130:33505)
0806 00:51:21.589300 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:21.589308 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:21.591393 (+ 2085us) server_negotiation.cc:407] Connection header received
0806 00:51:21.591474 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:21.591480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:21.591665 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:21.591799 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:21.592224 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.593758 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.595185 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.596748 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.597549 (+ 801us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:21.597665 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:21.597714 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 00:51:21.597803 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":80,"threads_started":1}
I20260806 00:51:21.600072 2222 tablet_service.cc:1467] Processing CreateTablet for tablet 40119cbdfb8b4941ba0771d80b277ca0 (DEFAULT_TABLE table=test-table [id=b9256bcb1c1d4a7d906822bb56edf0e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:51:21.600533 2439 tablet_service.cc:1467] Processing CreateTablet for tablet 40119cbdfb8b4941ba0771d80b277ca0 (DEFAULT_TABLE table=test-table [id=b9256bcb1c1d4a7d906822bb56edf0e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:51:21.601280 2222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40119cbdfb8b4941ba0771d80b277ca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:21.601513 2439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40119cbdfb8b4941ba0771d80b277ca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:21.610523 2790 tablet_bootstrap.cc:492] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9: Bootstrap starting.
I20260806 00:51:21.612551 2788 tablet_bootstrap.cc:492] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174: Bootstrap starting.
I20260806 00:51:21.613103 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:21.589460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:46449 (local address 127.0.0.1:37280)
0806 00:51:21.590917 (+ 1457us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:21.590929 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:21.590998 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:21.602474 (+ 11476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:21.602478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:21.602496 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:21.602581 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:21.602587 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.610287 (+ 7700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.610290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:21.611540 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:21.611552 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.612721 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.612725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:21.612818 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:21.612855 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 00:51:21.612901 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 00:51:21.612971 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1401,"thread_start_us":86,"threads_started":1}
I20260806 00:51:21.613972 2786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:21.589634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37280 (local address 127.15.147.131:46449)
0806 00:51:21.602009 (+ 12375us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:21.602016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:21.602039 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:51:21.602114 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:21.602118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:21.602297 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:21.602737 (+ 440us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:21.602906 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.610144 (+ 7238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.611689 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.612548 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.613557 (+ 1009us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:21.613680 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:21.613748 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 00:51:21.613831 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12262,"thread_start_us":82,"threads_started":1}
I20260806 00:51:21.614688 2788 tablet_bootstrap.cc:654] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:21.615569 2788 log.cc:824] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:21.616492 2790 tablet_bootstrap.cc:654] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:21.617297 2790 log.cc:824] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:21.616823 2626 tablet_service.cc:1467] Processing CreateTablet for tablet 40119cbdfb8b4941ba0771d80b277ca0 (DEFAULT_TABLE table=test-table [id=b9256bcb1c1d4a7d906822bb56edf0e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:51:21.618569 2626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40119cbdfb8b4941ba0771d80b277ca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:21.620043 2788 tablet_bootstrap.cc:492] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174: No bootstrap required, opened a new log
I20260806 00:51:21.620260 2788 ts_tablet_manager.cc:1397] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 00:51:21.625077 2788 raft_consensus.cc:348] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85611f732dad42a19e452e4d73568174" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38655 } } peers { permanent_uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33505 } } peers { permanent_uuid: "31f99b9db8a7499cbab22bf1e3b33f83" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46449 } }
I20260806 00:51:21.625360 2788 raft_consensus.cc:374] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:51:21.625428 2788 raft_consensus.cc:729] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85611f732dad42a19e452e4d73568174, State: Initialized, Role: FOLLOWER
I20260806 00:51:21.627692 2790 tablet_bootstrap.cc:492] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9: No bootstrap required, opened a new log
I20260806 00:51:21.627935 2790 ts_tablet_manager.cc:1397] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.000s
I20260806 00:51:21.632171 2790 raft_consensus.cc:348] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85611f732dad42a19e452e4d73568174" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38655 } } peers { permanent_uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33505 } } peers { permanent_uuid: "31f99b9db8a7499cbab22bf1e3b33f83" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46449 } }
I20260806 00:51:21.632520 2790 raft_consensus.cc:374] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:51:21.632597 2790 raft_consensus.cc:729] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa5c2f9a80ea4fb8b1f4d0d8ea358fb9, State: Initialized, Role: FOLLOWER
I20260806 00:51:21.632817 2788 consensus_queue.cc:260] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85611f732dad42a19e452e4d73568174" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38655 } } peers { permanent_uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33505 } } peers { permanent_uuid: "31f99b9db8a7499cbab22bf1e3b33f83" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46449 } }
I20260806 00:51:21.633082 2790 consensus_queue.cc:260] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85611f732dad42a19e452e4d73568174" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38655 } } peers { permanent_uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33505 } } peers { permanent_uuid: "31f99b9db8a7499cbab22bf1e3b33f83" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46449 } }
I20260806 00:51:21.634024 2788 ts_tablet_manager.cc:1428] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260806 00:51:21.634087 2790 ts_tablet_manager.cc:1428] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:51:21.634485 2343 heartbeater.cc:503] Master 127.15.147.190:43551 was elected leader, sending a full tablet report...
I20260806 00:51:21.634487 2536 heartbeater.cc:503] Master 127.15.147.190:43551 was elected leader, sending a full tablet report...
I20260806 00:51:21.634889 2794 tablet_bootstrap.cc:492] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83: Bootstrap starting.
I20260806 00:51:21.638414 2794 tablet_bootstrap.cc:654] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:21.639343 2794 log.cc:824] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:21.641686 2794 tablet_bootstrap.cc:492] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83: No bootstrap required, opened a new log
I20260806 00:51:21.641934 2794 ts_tablet_manager.cc:1397] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 00:51:21.646256 2794 raft_consensus.cc:348] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85611f732dad42a19e452e4d73568174" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38655 } } peers { permanent_uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33505 } } peers { permanent_uuid: "31f99b9db8a7499cbab22bf1e3b33f83" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46449 } }
I20260806 00:51:21.646546 2794 raft_consensus.cc:374] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:51:21.646602 2794 raft_consensus.cc:729] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31f99b9db8a7499cbab22bf1e3b33f83, State: Initialized, Role: FOLLOWER
I20260806 00:51:21.647086 2794 consensus_queue.cc:260] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85611f732dad42a19e452e4d73568174" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38655 } } peers { permanent_uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33505 } } peers { permanent_uuid: "31f99b9db8a7499cbab22bf1e3b33f83" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46449 } }
I20260806 00:51:21.647799 2753 heartbeater.cc:503] Master 127.15.147.190:43551 was elected leader, sending a full tablet report...
I20260806 00:51:21.648214 2794 ts_tablet_manager.cc:1428] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260806 00:51:21.732690 2344 tablet.cc:2369] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 00:51:21 dist-test-slave-45dx krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977481, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for HTTP/127.15.147.148@KRBTEST.COM
W20260806 00:51:21.773782 2754 tablet.cc:2369] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:51:21.774631 2537 tablet.cc:2369] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:51:21.793434 2795 raft_consensus.cc:482] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:51:21.793676 2795 raft_consensus.cc:504] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85611f732dad42a19e452e4d73568174" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38655 } } peers { permanent_uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33505 } } peers { permanent_uuid: "31f99b9db8a7499cbab22bf1e3b33f83" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46449 } }
I20260806 00:51:21.795071 2795 leader_election.cc:288] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 (127.15.147.130:33505), 31f99b9db8a7499cbab22bf1e3b33f83 (127.15.147.131:46449)
I20260806 00:51:21.798580 2798 raft_consensus.cc:482] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:51:21.798780 2798 raft_consensus.cc:504] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85611f732dad42a19e452e4d73568174" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38655 } } peers { permanent_uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33505 } } peers { permanent_uuid: "31f99b9db8a7499cbab22bf1e3b33f83" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46449 } }
I20260806 00:51:21.800042 2798 leader_election.cc:288] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85611f732dad42a19e452e4d73568174 (127.15.147.129:38655), aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 (127.15.147.130:33505)
I20260806 00:51:21.801921 2784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:21.795868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:59709 (local address 127.15.147.130:33505)
0806 00:51:21.795991 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:21.795995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:21.796010 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:51:21.796139 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:21.796142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:21.796189 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:21.796234 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:21.796707 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.797527 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.799723 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.800362 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.800456 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:21.800514 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:21.801658 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0806 00:51:21.801735 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:51:21.802546 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:21.795398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:33505 (local address 127.15.147.129:59709)
0806 00:51:21.795875 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:21.795892 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:21.796006 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:21.796409 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:21.796412 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:21.796432 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:21.796572 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:21.796578 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.798057 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.798061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:21.799574 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:21.799586 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.801413 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.801416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:21.801516 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:21.801539 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:51:21.802314 (+ 775us) client_negotiation.cc:240] Negotiation successful
0806 00:51:21.802404 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":122,"threads_started":1}
I20260806 00:51:21.803340 2471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40119cbdfb8b4941ba0771d80b277ca0" candidate_uuid: "85611f732dad42a19e452e4d73568174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9" is_pre_election: true
I20260806 00:51:21.803648 2471 raft_consensus.cc:2454] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85611f732dad42a19e452e4d73568174 in term 0.
I20260806 00:51:21.804270 2175 leader_election.cc:302] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85611f732dad42a19e452e4d73568174, aa5c2f9a80ea4fb8b1f4d0d8ea358fb9; no voters:
I20260806 00:51:21.804615 2795 raft_consensus.cc:2781] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:51:21.804715 2795 raft_consensus.cc:482] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:51:21.804757 2795 raft_consensus.cc:3037] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:21.806098 2795 raft_consensus.cc:504] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85611f732dad42a19e452e4d73568174" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38655 } } peers { permanent_uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33505 } } peers { permanent_uuid: "31f99b9db8a7499cbab22bf1e3b33f83" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46449 } }
I20260806 00:51:21.806459 2795 leader_election.cc:288] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [CANDIDATE]: Term 1 election: Requested vote from peers aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 (127.15.147.130:33505), 31f99b9db8a7499cbab22bf1e3b33f83 (127.15.147.131:46449)
I20260806 00:51:21.807117 2786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:21.795567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:55737 (local address 127.15.147.131:46449)
0806 00:51:21.795700 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:21.795705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:21.802934 (+ 7229us) server_negotiation.cc:407] Connection header received
0806 00:51:21.803136 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:21.803140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:21.803188 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:21.803233 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:21.803579 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.804291 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.805766 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.806363 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.806460 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:21.806517 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:21.806915 (+ 398us) server_negotiation.cc:299] Negotiation successful
0806 00:51:21.806996 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:51:21.807113 2471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40119cbdfb8b4941ba0771d80b277ca0" candidate_uuid: "85611f732dad42a19e452e4d73568174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9"
I20260806 00:51:21.807230 2471 raft_consensus.cc:3037] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:21.807674 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:21.795399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:46449 (local address 127.15.147.129:55737)
0806 00:51:21.802859 (+ 7460us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:21.802874 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:21.803017 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:21.803369 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:21.803373 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:21.803389 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:21.803456 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:21.803461 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.804466 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.804469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:21.805626 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:21.805637 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.806639 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.806642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:21.806762 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:21.806784 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:51:21.807459 (+ 675us) client_negotiation.cc:240] Negotiation successful
0806 00:51:21.807537 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7403,"mutex_wait_us":18,"thread_start_us":143,"threads_started":1}
I20260806 00:51:21.808518 2471 raft_consensus.cc:2454] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85611f732dad42a19e452e4d73568174 in term 1.
I20260806 00:51:21.808593 2671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40119cbdfb8b4941ba0771d80b277ca0" candidate_uuid: "85611f732dad42a19e452e4d73568174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31f99b9db8a7499cbab22bf1e3b33f83" is_pre_election: true
I20260806 00:51:21.808782 2671 raft_consensus.cc:2454] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85611f732dad42a19e452e4d73568174 in term 0.
I20260806 00:51:21.808840 2175 leader_election.cc:302] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85611f732dad42a19e452e4d73568174, aa5c2f9a80ea4fb8b1f4d0d8ea358fb9; no voters:
I20260806 00:51:21.808991 2795 raft_consensus.cc:2781] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:51:21.809049 2670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40119cbdfb8b4941ba0771d80b277ca0" candidate_uuid: "85611f732dad42a19e452e4d73568174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31f99b9db8a7499cbab22bf1e3b33f83"
I20260806 00:51:21.809149 2670 raft_consensus.cc:3037] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:21.809216 2795 raft_consensus.cc:686] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [term 1 LEADER]: Becoming Leader. State: Replica: 85611f732dad42a19e452e4d73568174, State: Running, Role: LEADER
I20260806 00:51:21.809583 2795 consensus_queue.cc:237] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85611f732dad42a19e452e4d73568174" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38655 } } peers { permanent_uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33505 } } peers { permanent_uuid: "31f99b9db8a7499cbab22bf1e3b33f83" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46449 } }
I20260806 00:51:21.810351 2670 raft_consensus.cc:2454] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85611f732dad42a19e452e4d73568174 in term 1.
I20260806 00:51:21.812866 2054 catalog_manager.cc:5495] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 reported cstate change: term changed from 0 to 1, leader changed from <none> to 85611f732dad42a19e452e4d73568174 (127.15.147.129). New cstate: current_term: 1 leader_uuid: "85611f732dad42a19e452e4d73568174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85611f732dad42a19e452e4d73568174" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31f99b9db8a7499cbab22bf1e3b33f83" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46449 } health_report { overall_health: UNKNOWN } } }
I20260806 00:51:21.815888 2783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:21.800414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:42941 (local address 127.15.147.129:38655)
0806 00:51:21.800523 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:21.800528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:21.811379 (+ 10851us) server_negotiation.cc:407] Connection header received
0806 00:51:21.811505 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:21.811509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:21.811553 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:21.811626 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:21.811894 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.812622 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.814099 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.814661 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.814747 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:21.814788 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:21.815715 (+ 927us) server_negotiation.cc:299] Negotiation successful
0806 00:51:21.815783 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 00:51:21.816265 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:21.800339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:38655 (local address 127.15.147.131:42941)
0806 00:51:21.811285 (+ 10946us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:21.811299 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:21.811418 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:21.811634 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:21.811637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:21.811652 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:21.811763 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:21.811769 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.812750 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.812753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:21.813961 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:21.813972 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.814881 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.814884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:21.815562 (+ 678us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:21.815588 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 00:51:21.816052 (+ 464us) client_negotiation.cc:240] Negotiation successful
0806 00:51:21.816125 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10879,"thread_start_us":130,"threads_started":1}
I20260806 00:51:21.816841 2255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40119cbdfb8b4941ba0771d80b277ca0" candidate_uuid: "31f99b9db8a7499cbab22bf1e3b33f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85611f732dad42a19e452e4d73568174" is_pre_election: true
I20260806 00:51:21.821156 2784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:21.800750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:52571 (local address 127.15.147.130:33505)
0806 00:51:21.802075 (+ 1325us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:21.802081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:21.811865 (+ 9784us) server_negotiation.cc:407] Connection header received
0806 00:51:21.812055 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:21.812058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:21.812099 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:21.812142 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:21.817819 (+ 5677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.818525 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.819904 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.820468 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.820555 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:21.820598 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:21.820947 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 00:51:21.821024 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1269,"thread_start_us":118,"threads_started":1}
I20260806 00:51:21.821589 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:21.800339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:33505 (local address 127.15.147.131:52571)
0806 00:51:21.811776 (+ 11437us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:21.811790 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:21.811936 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:21.812250 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:21.812253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:21.812267 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:21.817683 (+ 5416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:21.817693 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.818658 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.818661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:21.819748 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:21.819759 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.820693 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.820696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:21.820802 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:21.820823 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 00:51:21.821364 (+ 541us) client_negotiation.cc:240] Negotiation successful
0806 00:51:21.821432 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11372,"mutex_wait_us":12,"thread_start_us":119,"threads_started":1}
I20260806 00:51:21.829458 2471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40119cbdfb8b4941ba0771d80b277ca0" candidate_uuid: "31f99b9db8a7499cbab22bf1e3b33f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9" is_pre_election: true
I20260806 00:51:21.829686 2471 raft_consensus.cc:2379] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31f99b9db8a7499cbab22bf1e3b33f83 in current term 1: Already voted for candidate 85611f732dad42a19e452e4d73568174 in this term.
I20260806 00:51:21.830247 2570 leader_election.cc:302] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31f99b9db8a7499cbab22bf1e3b33f83; no voters: 85611f732dad42a19e452e4d73568174, aa5c2f9a80ea4fb8b1f4d0d8ea358fb9
I20260806 00:51:21.830777 2798 raft_consensus.cc:2726] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 00:51:21.843772 2783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:21.838039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47654 (local address 127.15.147.129:38655)
0806 00:51:21.838159 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:21.838164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:21.838179 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:51:21.838266 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:21.838272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:21.838376 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:21.838499 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:51:21.838898 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.839603 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.841520 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:21.841899 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:21.842000 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:21.842438 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:51:21.843039 (+ 601us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:51:21.843160 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:21.843569 (+ 409us) server_negotiation.cc:299] Negotiation successful
0806 00:51:21.843626 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 00:51:21.855119 2471 raft_consensus.cc:1261] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 [term 1 FOLLOWER]: Refusing update from remote peer 85611f732dad42a19e452e4d73568174: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:51:21.855298 2670 raft_consensus.cc:1261] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83 [term 1 FOLLOWER]: Refusing update from remote peer 85611f732dad42a19e452e4d73568174: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:51:21.855867 2795 consensus_queue.cc:1035] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:51:21.856110 2810 consensus_queue.cc:1035] T 40119cbdfb8b4941ba0771d80b277ca0 P 85611f732dad42a19e452e4d73568174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31f99b9db8a7499cbab22bf1e3b33f83" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 46449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:51:21.879469 2815 mvcc.cc:205] Tried to move back new op lower bound from 7315363765664645120 to 7315363765494996992. Current Snapshot: MvccSnapshot[applied={T|T < 7315363765664645120}]
I20260806 00:51:21.879910 2816 mvcc.cc:205] Tried to move back new op lower bound from 7315363765664645120 to 7315363765494996992. Current Snapshot: MvccSnapshot[applied={T|T < 7315363765664645120}]
I20260806 00:51:21.898018 2054 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:51:21.898205 2054 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:51:21.901407 2054 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b6d7603a60e4119ac9a0db6a09058db: Sending DeleteTablet for 3 replicas of tablet 40119cbdfb8b4941ba0771d80b277ca0
I20260806 00:51:21.902298 2222 tablet_service.cc:1514] Processing DeleteTablet for tablet 40119cbdfb8b4941ba0771d80b277ca0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:51:21 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:47648
I20260806 00:51:21.913057 2439 tablet_service.cc:1514] Processing DeleteTablet for tablet 40119cbdfb8b4941ba0771d80b277ca0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:51:21 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:56316
I20260806 00:51:21.914250 2626 tablet_service.cc:1514] Processing DeleteTablet for tablet 40119cbdfb8b4941ba0771d80b277ca0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:51:21 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:37280
I20260806 00:51:21.915220 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 2110
I20260806 00:51:21.929451 2829 tablet_replica.cc:330] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9: stopping tablet replica
I20260806 00:51:21.929956 2829 raft_consensus.cc:2229] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:51:21.930641 2829 pending_rounds.cc:62] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9: Trying to abort 2 pending ops.
I20260806 00:51:21.930800 2829 pending_rounds.cc:68] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7315363765664645120 op_type: WRITE_OP write_request { tablet_id: "40119cbdfb8b4941ba0771d80b277ca0" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\265\267\317\323\006\"8\n\ntest-admin\022*\n b9256bcb1c1d4a7d906822bb56edf0e0\020\001\030\001 \001(\001" signature: "<redacted>""\244y\201\213\364\372\207m\354\227\372\225U\001/HI\251\230`$\215B\030\262\375l\335\350t\224A\376\334DE#xu#\337J\020r\0251\\\t\335Uq!\332\302\201\336\245\353y\222\2603t\341" signing_key_seq_num: 0 } } request_id { client_id: "e9c76a39e5df46f18cc4a8f2dd47daba" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260806 00:51:21.932883 2829 pending_rounds.cc:68] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315363765494996992 op_type: NO_OP noop_request { }
I20260806 00:51:21.933131 2829 raft_consensus.cc:2866] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260806 00:51:21.933269 2829 raft_consensus.cc:2258] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 00:51:21.938210 2821 consensus_queue.cc:1159] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "aa5c2f9a80ea4fb8b1f4d0d8ea358fb9" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260806 00:51:21.939961 2829 ts_tablet_manager.cc:1905] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:51:21.942960 2829 ts_tablet_manager.cc:1918] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:51:21.943140 2829 log.cc:1196] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/40119cbdfb8b4941ba0771d80b277ca0
I20260806 00:51:21.943557 2829 ts_tablet_manager.cc:1939] T 40119cbdfb8b4941ba0771d80b277ca0 P aa5c2f9a80ea4fb8b1f4d0d8ea358fb9: Deleting consensus metadata
I20260806 00:51:21.944562 2830 tablet_replica.cc:330] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83: stopping tablet replica
I20260806 00:51:21.949863 2039 catalog_manager.cc:4841] TS aa5c2f9a80ea4fb8b1f4d0d8ea358fb9 (127.15.147.130:33505): tablet 40119cbdfb8b4941ba0771d80b277ca0 (table test-table [id=b9256bcb1c1d4a7d906822bb56edf0e0]) successfully deleted
I20260806 00:51:21.950496 2830 raft_consensus.cc:2229] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:51:21.950930 2830 raft_consensus.cc:2258] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 00:51:21.952530 2380 connection.cc:535] server connection from 127.15.147.129:59709 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 00:51:21.952965 2830 ts_tablet_manager.cc:1905] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 00:51:21.953723 2041 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.147.129:38655 (error 108)
I20260806 00:51:21.954710 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 2349
I20260806 00:51:21.959071 2830 ts_tablet_manager.cc:1918] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:51:21.959177 2830 log.cc:1196] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/40119cbdfb8b4941ba0771d80b277ca0
I20260806 00:51:21.959499 2830 ts_tablet_manager.cc:1939] T 40119cbdfb8b4941ba0771d80b277ca0 P 31f99b9db8a7499cbab22bf1e3b33f83: Deleting consensus metadata
I20260806 00:51:21.961037 2038 catalog_manager.cc:4841] TS 31f99b9db8a7499cbab22bf1e3b33f83 (127.15.147.131:46449): tablet 40119cbdfb8b4941ba0771d80b277ca0 (table test-table [id=b9256bcb1c1d4a7d906822bb56edf0e0]) successfully deleted
I20260806 00:51:21.962950 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:21.962568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:38655 (local address 127.0.0.1:47660)
0806 00:51:21.962771 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:21.962855 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:38655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260806 00:51:21.963263 2041 catalog_manager.cc:4584] TS 85611f732dad42a19e452e4d73568174 (127.15.147.129:38655): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 40119cbdfb8b4941ba0771d80b277ca0: Network error: Client connection negotiation failed: client connection to 127.15.147.129:38655: connect: Connection refused (error 111)
I20260806 00:51:21.970091 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 2540
I20260806 00:51:22.017045 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:22.016647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:38655 (local address 127.0.0.1:47672)
0806 00:51:22.016861 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:22.016927 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:38655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260806 00:51:22.018013 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:22.017745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:38655 (local address 127.0.0.1:47674)
0806 00:51:22.017888 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:22.017936 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:38655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 00:51:22.025354 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 2007
I20260806 00:51:22.038589 15950 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 00:51:22.385438 15950 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 00:51:22.385555 15950 mini_ranger.cc:66] Stopping Ranger...
I20260806 00:51:22.730507 15950 mini_ranger.cc:68] Stopped Ranger
2026-08-06 00:51:22.730 UTC [705] LOG: received smart shutdown request
2026-08-06 00:51:22.735 UTC [705] LOG: background worker "logical replication launcher" (PID 721) exited with exit code 1
2026-08-06 00:51:22.735 UTC [712] LOG: shutting down
2026-08-06 00:51:22.888 UTC [705] LOG: database system is shut down
2026-08-06T00:51:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50273 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:51:23 dist-test-slave-45dx krb5kdc[2845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:51:23 dist-test-slave-45dx krb5kdc[2845](info): set up 2 sockets
Aug 06 00:51:23 dist-test-slave-45dx krb5kdc[2845](info): commencing operation
krb5kdc: starting...
W20260806 00:51:24.217013 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:51:24 dist-test-slave-45dx krb5kdc[2845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:51:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:51:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:51:24.347270 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:46115
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:37887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:46115,127.15.147.189:34965,127.15.147.188:43987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:51:24.457619 2864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:24.457901 2864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:24.457973 2864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:24.461798 2864 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:51:24.461884 2864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:24.461921 2864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:24.462004 2864 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:51:24.462042 2864 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:51:24.467010 2864 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:37887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:46115,127.15.147.189:34965,127.15.147.188:43987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:46115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:24.468001 2864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:24.469030 2864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:24.478035 2873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:24.478072 2872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:24.478206 2864 server_base.cc:1034] running on GCE node
W20260806 00:51:24.478096 2875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:24.478938 2864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:24.479939 2864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:24.481142 2864 hybrid_clock.cc:648] HybridClock initialized: now 1785977484481115 us; error 40 us; skew 500 ppm
Aug 06 00:51:24 dist-test-slave-45dx krb5kdc[2845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977484, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:24.484076 2864 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:51:24.484959 2864 webserver.cc:460] Webserver started at http://127.15.147.190:34487/ using document root <none> and password file <none>
I20260806 00:51:24.485532 2864 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:24.485627 2864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:24.485908 2864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:51:24.488085 2864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "fa4135655a334868b351d82525ad0ced"
format_stamp: "Formatted at 2026-08-06 00:51:24 on dist-test-slave-45dx"
server_key: "2675d553e56f61f2571d563eb22a47a9"
server_key_iv: "017f8b346d3fa6c07c019e045fc9a47b"
server_key_version: "encryptionkey@0"
I20260806 00:51:24.488688 2864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "fa4135655a334868b351d82525ad0ced"
format_stamp: "Formatted at 2026-08-06 00:51:24 on dist-test-slave-45dx"
server_key: "2675d553e56f61f2571d563eb22a47a9"
server_key_iv: "017f8b346d3fa6c07c019e045fc9a47b"
server_key_version: "encryptionkey@0"
I20260806 00:51:24.492218 2864 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:51:24.494521 2886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:24.495700 2864 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:51:24.495868 2864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "fa4135655a334868b351d82525ad0ced"
format_stamp: "Formatted at 2026-08-06 00:51:24 on dist-test-slave-45dx"
server_key: "2675d553e56f61f2571d563eb22a47a9"
server_key_iv: "017f8b346d3fa6c07c019e045fc9a47b"
server_key_version: "encryptionkey@0"
I20260806 00:51:24.495985 2864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:24.545624 2864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:24.548659 2864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:24.548880 2864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:24.555927 2864 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:46115
I20260806 00:51:24.556205 2960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:46115 every 8 connection(s)
I20260806 00:51:24.556910 2864 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:51:24.559592 2961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:24.563316 2961 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:51:24.564927 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 2864
I20260806 00:51:24.565053 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:51:24.565363 15950 external_mini_cluster.cc:1428] Setting key 0c5fff79cf454bd87d377c1498006d83
Aug 06 00:51:24 dist-test-slave-45dx krb5kdc[2845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977484, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.147.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.189@KRBTEST.COM" created.
I20260806 00:51:24.578641 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.565705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54538 (local address 127.15.147.190:46115)
0806 00:51:24.566124 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:24.566145 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:24.566428 (+ 283us) server_negotiation.cc:407] Connection header received
0806 00:51:24.567450 (+ 1022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:24.567455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:24.567638 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:24.567827 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:24.568771 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.569677 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.570773 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.571136 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.571326 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:24.573726 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:24.573731 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:24.573741 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:24.573761 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:24.575465 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:24.575949 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:24.575955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:24.575956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:24.576067 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:24.576406 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:24.576409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:24.576410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:24.576764 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:24.576904 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:24.577299 (+ 395us) server_negotiation.cc:299] Negotiation successful
0806 00:51:24.577545 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":142,"threads_started":1}
I20260806 00:51:24.578641 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.565706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:46115 (local address 127.0.0.1:54538)
0806 00:51:24.566286 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.566323 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:24.567138 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:24.567826 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:24.567831 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:24.568332 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:24.568613 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:24.568628 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.569832 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.569835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:24.570657 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:24.570667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.571313 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.571316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:24.571428 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:24.572049 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:24.572066 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:24.573565 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:24.575599 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:24.575609 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:24.575618 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:24.575841 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:24.576162 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:24.576166 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:24.576168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:24.576282 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:24.576886 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:24.576891 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:24.577055 (+ 164us) client_negotiation.cc:769] Sending connection context
0806 00:51:24.577218 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 00:51:24.577600 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":94,"thread_start_us":146,"threads_started":1}
I20260806 00:51:24.581676 2961 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:51:24.582446 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.582178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:34965 (local address 127.0.0.1:57574)
0806 00:51:24.582302 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.582381 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 00:51:24.582713 2894 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34965: connect: Connection refused (error 111)
I20260806 00:51:24.583649 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.583427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:34965 (local address 127.0.0.1:57586)
0806 00:51:24.583555 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.583598 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 00:51:24.583858 2961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.189:34965: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34965: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.189.keytab.
Entry for principal kudu/127.15.147.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.189.keytab.
I20260806 00:51:24.594663 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.189:34965
--webserver_interface=127.15.147.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.189.keytab
--principal=kudu/127.15.147.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:37887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:46115,127.15.147.189:34965,127.15.147.188:43987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:51:24.633101 2961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } attempt: 1
I20260806 00:51:24.634047 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.633734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:34965 (local address 127.0.0.1:57592)
0806 00:51:24.633910 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.633970 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 00:51:24.634750 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.634567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:34965 (local address 127.0.0.1:57608)
0806 00:51:24.634662 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.634703 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 00:51:24.634938 2961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.189:34965: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34965: connect: Connection refused (error 111)
I20260806 00:51:24.703118 2961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } attempt: 2
I20260806 00:51:24.703940 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.703623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:34965 (local address 127.0.0.1:57622)
0806 00:51:24.703793 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.703861 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 00:51:24.704703 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.704495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:34965 (local address 127.0.0.1:57638)
0806 00:51:24.704602 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.704653 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 00:51:24.704856 2961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.189:34965: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34965: connect: Connection refused (error 111)
W20260806 00:51:24.706421 2972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:24.706630 2972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:24.706686 2972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:24.710645 2972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:51:24.710695 2972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:24.710712 2972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:24.710747 2972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:51:24.710762 2972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:51:24.715947 2972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:37887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:46115,127.15.147.189:34965,127.15.147.188:43987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.189.keytab
--principal=kudu/127.15.147.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.189:34965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.147.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:24.717015 2972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:24.718001 2972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:24.726435 2980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:24.726536 2972 server_base.cc:1034] running on GCE node
W20260806 00:51:24.726441 2982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:24.726498 2979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:24.727267 2972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:24.728436 2972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:24.729653 2972 hybrid_clock.cc:648] HybridClock initialized: now 1785977484729636 us; error 29 us; skew 500 ppm
Aug 06 00:51:24 dist-test-slave-45dx krb5kdc[2845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977484, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:24.732813 2972 init.cc:377] Logged in from keytab as kudu/127.15.147.189@KRBTEST.COM (short username kudu)
I20260806 00:51:24.733656 2972 webserver.cc:460] Webserver started at http://127.15.147.189:43955/ using document root <none> and password file <none>
I20260806 00:51:24.734164 2972 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:24.734222 2972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:24.734383 2972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:51:24.736373 2972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-1/data/instance:
uuid: "7eea17f57aab4eafa7ee9fac5602ef8f"
format_stamp: "Formatted at 2026-08-06 00:51:24 on dist-test-slave-45dx"
server_key: "fb7637ab0f870b601d7205e844dfc057"
server_key_iv: "5e373d5e25908ade0f28161a85180231"
server_key_version: "encryptionkey@0"
I20260806 00:51:24.736925 2972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-1/wal/instance:
uuid: "7eea17f57aab4eafa7ee9fac5602ef8f"
format_stamp: "Formatted at 2026-08-06 00:51:24 on dist-test-slave-45dx"
server_key: "fb7637ab0f870b601d7205e844dfc057"
server_key_iv: "5e373d5e25908ade0f28161a85180231"
server_key_version: "encryptionkey@0"
I20260806 00:51:24.740669 2972 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:51:24.743026 2990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:24.744172 2972 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:51:24.744299 2972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-1/wal
uuid: "7eea17f57aab4eafa7ee9fac5602ef8f"
format_stamp: "Formatted at 2026-08-06 00:51:24 on dist-test-slave-45dx"
server_key: "fb7637ab0f870b601d7205e844dfc057"
server_key_iv: "5e373d5e25908ade0f28161a85180231"
server_key_version: "encryptionkey@0"
I20260806 00:51:24.744447 2972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:24.762662 2972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:24.765879 2972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:24.766084 2972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:24.773279 2972 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.189:34965
I20260806 00:51:24.773586 3068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.189:34965 every 8 connection(s)
I20260806 00:51:24.774224 2972 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-1/data/info.pb
I20260806 00:51:24.776965 3069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:24.780501 3069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:51:24.782087 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 2972
I20260806 00:51:24.782229 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-1/wal/instance
I20260806 00:51:24.782496 15950 external_mini_cluster.cc:1428] Setting key d15c1d8125ad214a37582fc26ef5ea7d
Aug 06 00:51:24 dist-test-slave-45dx krb5kdc[2845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977484, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.189@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.147.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.188@KRBTEST.COM" created.
I20260806 00:51:24.794661 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.782266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54542 (local address 127.15.147.190:46115)
0806 00:51:24.782405 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:24.782410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:24.783332 (+ 922us) server_negotiation.cc:407] Connection header received
0806 00:51:24.784615 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:24.784621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:24.784688 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:24.784787 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:24.785831 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.786653 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.787864 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.788284 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.788439 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:24.791172 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:24.791180 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:24.791182 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:24.791205 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:24.792660 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:24.793140 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:24.793144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:24.793146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:24.793228 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:24.793556 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:24.793559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:24.793561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:24.793731 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:24.793832 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:24.794366 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 00:51:24.794492 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:51:24.795538 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.782672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:46115 (local address 127.0.0.1:54542)
0806 00:51:24.783154 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.783208 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:24.784352 (+ 1144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:24.784945 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:24.784951 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:24.785256 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:24.785683 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:24.785695 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.786781 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.786783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:24.787731 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:24.787741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.788503 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.788510 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:24.788744 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:24.789371 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:24.789395 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:24.791016 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:24.792776 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:24.792782 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:24.792794 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:24.793038 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:24.793318 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:24.793321 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:24.793323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:24.793456 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:24.793846 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:24.793853 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:24.794098 (+ 245us) client_negotiation.cc:769] Sending connection context
0806 00:51:24.794306 (+ 208us) client_negotiation.cc:240] Negotiation successful
0806 00:51:24.794616 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":105,"threads_started":1}
I20260806 00:51:24.796021 2961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } attempt: 3
I20260806 00:51:24.797161 3069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:51:24 dist-test-slave-45dx krb5kdc[2845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977484, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.189@KRBTEST.COM for kudu/127.15.147.189@KRBTEST.COM
Aug 06 00:51:24 dist-test-slave-45dx krb5kdc[2845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977484, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for kudu/127.15.147.189@KRBTEST.COM
I20260806 00:51:24.806710 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.797668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:34965 (local address 127.0.0.1:57656)
0806 00:51:24.797822 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.797836 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:24.797900 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:24.798395 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:24.798401 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:24.798602 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:24.798738 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:24.798749 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.799748 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.799753 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:24.800433 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:24.800446 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.801003 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.801006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:24.801094 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:24.801593 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:24.801607 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:24.803054 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:24.805007 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:24.805010 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:24.805012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:24.805197 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:24.805522 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:24.805526 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:24.805528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:24.805610 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:24.806156 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:24.806159 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:24.806261 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 00:51:24.806379 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 00:51:24.806529 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 00:51:24.806862 3078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.797740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57656 (local address 127.15.147.189:34965)
0806 00:51:24.798129 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:24.798139 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:24.798150 (+ 11us) server_negotiation.cc:407] Connection header received
0806 00:51:24.798179 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:24.798181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:24.798241 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:24.798326 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:24.798885 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.799666 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.800547 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.800884 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.800998 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:24.803172 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:24.803181 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:24.803192 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:24.803213 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:24.804891 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:24.805330 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:24.805335 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:24.805337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:24.805423 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:24.805704 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:24.805709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:24.805712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:24.806041 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:24.806215 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:24.806476 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 00:51:24.806640 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":311,"spinlock_wait_cycles":4608,"thread_start_us":93,"threads_started":1}
I20260806 00:51:24.808583 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.796514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:34965 (local address 127.0.0.1:57650)
0806 00:51:24.797121 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.797139 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:24.797217 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:24.797697 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:24.797700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:24.797881 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:24.798022 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:24.798032 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.799409 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.799415 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:24.801154 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:24.801169 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.802100 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.802105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:24.802225 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:24.802693 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:24.802708 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:24.804167 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:24.806924 (+ 2757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:24.806927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:24.806929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:24.807202 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:24.807533 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:24.807537 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:24.807538 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:24.807638 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:24.808095 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:24.808097 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:24.808177 (+ 80us) client_negotiation.cc:769] Sending connection context
0806 00:51:24.808282 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 00:51:24.808472 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524}
I20260806 00:51:24.808643 3077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.796833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57650 (local address 127.15.147.189:34965)
0806 00:51:24.797254 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:24.797267 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:24.797290 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:51:24.797334 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:24.797339 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:24.797539 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:24.798169 (+ 630us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:24.798325 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.799257 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.801588 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.801959 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.802084 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:24.804346 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:24.804352 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:24.804358 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:24.804388 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:24.806774 (+ 2386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:24.807363 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:24.807365 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:24.807366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:24.807437 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:24.807757 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:24.807760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:24.807762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:24.807989 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:24.808089 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:24.808396 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 00:51:24.808498 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"spinlock_wait_cycles":768,"thread_start_us":155,"threads_started":1}
I20260806 00:51:24.809015 3069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:51:24.809319 2961 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:51:24.809708 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.809450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:43987 (local address 127.0.0.1:37024)
0806 00:51:24.809534 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.809622 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 00:51:24.809966 2996 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
I20260806 00:51:24.810096 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.809854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:43987 (local address 127.0.0.1:37034)
0806 00:51:24.809959 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.810015 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 00:51:24.810806 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.810575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:43987 (local address 127.0.0.1:37048)
0806 00:51:24.810707 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.810751 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 00:51:24.810868 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.810625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:43987 (local address 127.0.0.1:37060)
0806 00:51:24.810730 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.810775 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 00:51:24.810954 2961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.188:43987: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
W20260806 00:51:24.810994 3069 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.188:43987: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.188.keytab.
Entry for principal kudu/127.15.147.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.188.keytab.
I20260806 00:51:24.813313 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.188:43987
--webserver_interface=127.15.147.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.188.keytab
--principal=kudu/127.15.147.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:37887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:46115,127.15.147.189:34965,127.15.147.188:43987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:51:24.842137 2961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } attempt: 1
I20260806 00:51:24.843019 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.842721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:43987 (local address 127.0.0.1:37074)
0806 00:51:24.842880 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.842941 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 00:51:24.843736 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.843512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:43987 (local address 127.0.0.1:37090)
0806 00:51:24.843623 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.843675 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 00:51:24.843873 2961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.188:43987: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
I20260806 00:51:24.860294 3069 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } attempt: 1
I20260806 00:51:24.861070 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.860813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:43987 (local address 127.0.0.1:37094)
0806 00:51:24.860949 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.861004 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 00:51:24.861764 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.861547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:43987 (local address 127.0.0.1:37106)
0806 00:51:24.861654 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.861703 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 00:51:24.861920 3069 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.188:43987: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
I20260806 00:51:24.919057 2961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } attempt: 2
I20260806 00:51:24.920001 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.919647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:43987 (local address 127.0.0.1:37120)
0806 00:51:24.919847 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.919916 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 00:51:24.920866 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.920623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:43987 (local address 127.0.0.1:37134)
0806 00:51:24.920742 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.920806 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 00:51:24.921011 2961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.188:43987: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
W20260806 00:51:24.926105 3080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:24.926362 3080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:24.926462 3080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 00:51:24.930087 3069 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } attempt: 2
W20260806 00:51:24.930423 3080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:51:24.930498 3080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:24.930560 3080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:24.930598 3080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:51:24.930634 3080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:51:24.930892 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.930598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:43987 (local address 127.0.0.1:37146)
0806 00:51:24.930757 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.930818 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 00:51:24.931636 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.931412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:43987 (local address 127.0.0.1:37158)
0806 00:51:24.931524 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.931576 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 00:51:24.931779 3069 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.188:43987: Network error: Client connection negotiation failed: client connection to 127.15.147.188:43987: connect: Connection refused (error 111)
I20260806 00:51:24.935943 3080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:37887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:46115,127.15.147.189:34965,127.15.147.188:43987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.188.keytab
--principal=kudu/127.15.147.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.188:43987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.147.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:24.936962 3080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:24.937968 3080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:24.946352 3085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:24.946393 3087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:24.946444 3080 server_base.cc:1034] running on GCE node
W20260806 00:51:24.946393 3092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:24.947177 3080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:24.948346 3080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:24.949535 3080 hybrid_clock.cc:648] HybridClock initialized: now 1785977484949522 us; error 34 us; skew 500 ppm
Aug 06 00:51:24 dist-test-slave-45dx krb5kdc[2845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977484, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:24.952178 3080 init.cc:377] Logged in from keytab as kudu/127.15.147.188@KRBTEST.COM (short username kudu)
I20260806 00:51:24.953034 3080 webserver.cc:460] Webserver started at http://127.15.147.188:33473/ using document root <none> and password file <none>
I20260806 00:51:24.953572 3080 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:24.953661 3080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:24.953884 3080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:51:24.955786 3080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-2/data/instance:
uuid: "01284c12eed949a497288848f75973bb"
format_stamp: "Formatted at 2026-08-06 00:51:24 on dist-test-slave-45dx"
server_key: "1bdca6696e73a72f6754836bd9ff94f8"
server_key_iv: "14ec5b8c1fb8ec4c9580562da58d1f80"
server_key_version: "encryptionkey@0"
I20260806 00:51:24.956375 3080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-2/wal/instance:
uuid: "01284c12eed949a497288848f75973bb"
format_stamp: "Formatted at 2026-08-06 00:51:24 on dist-test-slave-45dx"
server_key: "1bdca6696e73a72f6754836bd9ff94f8"
server_key_iv: "14ec5b8c1fb8ec4c9580562da58d1f80"
server_key_version: "encryptionkey@0"
I20260806 00:51:24.959952 3080 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:51:24.962260 3104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:24.963393 3080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:51:24.963539 3080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-2/wal
uuid: "01284c12eed949a497288848f75973bb"
format_stamp: "Formatted at 2026-08-06 00:51:24 on dist-test-slave-45dx"
server_key: "1bdca6696e73a72f6754836bd9ff94f8"
server_key_iv: "14ec5b8c1fb8ec4c9580562da58d1f80"
server_key_version: "encryptionkey@0"
I20260806 00:51:24.963647 3080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:24.977226 3080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:24.979803 3080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:24.980002 3080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:24.986931 3080 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.188:43987
I20260806 00:51:24.987234 3174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.188:43987 every 8 connection(s)
I20260806 00:51:24.987862 3080 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-2/data/info.pb
I20260806 00:51:24.989893 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 3080
I20260806 00:51:24.990015 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785977317010501-15950-0/minicluster-data/master-2/wal/instance
I20260806 00:51:24.990293 15950 external_mini_cluster.cc:1428] Setting key 31f68c4344598d054d7ea941f3d5bed2
I20260806 00:51:24.990790 3175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:24.995008 3175 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:51:24 dist-test-slave-45dx krb5kdc[2845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.188@KRBTEST.COM
Aug 06 00:51:24 dist-test-slave-45dx krb5kdc[2845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977484, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.188@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:51:25.005028 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.991588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37162 (local address 127.15.147.188:43987)
0806 00:51:24.992008 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:24.992022 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:24.992055 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:51:24.992749 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:24.992765 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:24.993063 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:24.993373 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:24.994207 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.995675 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.996591 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.997251 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.997419 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:25.000364 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:25.000373 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:25.000384 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:25.000416 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:25.002135 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.002587 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.002594 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.002601 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.002739 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.003068 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.003073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.003074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.003505 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:25.003700 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:25.003983 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 00:51:25.004269 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":137,"threads_started":1}
I20260806 00:51:25.006865 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.996810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:46115 (local address 127.0.0.1:54548)
0806 00:51:24.997251 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:24.997301 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:24.997458 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:24.997846 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:24.997852 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:24.998276 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:24.998404 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:24.998410 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.999409 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:24.999413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:25.000412 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:25.000429 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.000991 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.000994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:25.001078 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:25.001654 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:25.001666 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:25.003225 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:25.005062 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:25.005070 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:25.005076 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:25.005361 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:25.005684 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:25.005688 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:25.005689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:25.005813 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:25.006260 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:25.006268 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:25.006374 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 00:51:25.006517 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 00:51:25.006688 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":136,"threads_started":1}
I20260806 00:51:25.007112 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:24.996864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54548 (local address 127.15.147.190:46115)
0806 00:51:24.997016 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:24.997021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:24.997373 (+ 352us) server_negotiation.cc:407] Connection header received
0806 00:51:24.997620 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:24.997624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:24.997685 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:24.997785 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:24.998537 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:24.999286 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.000527 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.000872 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.000980 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:25.003362 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:25.003365 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:25.003367 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:25.003391 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:25.004946 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.005474 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.005483 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.005485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.005572 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.005933 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.005936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.005937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.006145 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:25.006257 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:25.006575 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 00:51:25.006688 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:51:25.008208 3175 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:51:25 dist-test-slave-45dx krb5kdc[2845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
Aug 06 00:51:25 dist-test-slave-45dx krb5kdc[2845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977484, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.188@KRBTEST.COM for kudu/127.15.147.189@KRBTEST.COM
I20260806 00:51:25.015949 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:25.007384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54562 (local address 127.15.147.190:46115)
0806 00:51:25.007491 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:25.007494 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:25.007506 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:51:25.007671 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:25.007673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:25.007741 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:25.007822 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:25.008548 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.009159 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.010070 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.010299 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.010396 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:25.012733 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:25.012736 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:25.012737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:25.012757 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:25.014353 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.014785 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.014787 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.014789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.014857 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.015142 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.015145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.015148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.015345 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:25.015538 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:25.015700 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 00:51:25.015812 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:51:25.016693 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:25.008674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:34965 (local address 127.0.0.1:57670)
0806 00:51:25.008807 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:25.008818 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:25.008877 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:25.009106 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:25.009108 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:25.009270 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:25.009366 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:25.009371 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.010265 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.010268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:25.010975 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:25.010991 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.011639 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.011641 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:25.011708 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:25.012128 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:25.012140 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:25.013268 (+ 1128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:25.015064 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:25.015066 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:25.015067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:25.015320 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:25.015646 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:25.015649 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:25.015650 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:25.015738 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:25.016252 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:25.016254 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:25.016403 (+ 149us) client_negotiation.cc:769] Sending connection context
0806 00:51:25.016498 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 00:51:25.016591 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 00:51:25.016856 3077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:25.008783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57670 (local address 127.15.147.189:34965)
0806 00:51:25.008924 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:25.008927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:25.008937 (+ 10us) server_negotiation.cc:407] Connection header received
0806 00:51:25.008975 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:25.008977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:25.009010 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:25.009069 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:25.009483 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.010170 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.011096 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.011532 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.011637 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:25.013418 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:25.013426 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:25.013428 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:25.013450 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:25.014967 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.015434 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.015437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.015439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.015527 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.015945 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.015947 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.015948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.016161 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:25.016262 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:25.016576 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 00:51:25.016707 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:51:25.017680 3175 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:51:25.020159 2961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } attempt: 3
Aug 06 00:51:25 dist-test-slave-45dx krb5kdc[2845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.189@KRBTEST.COM
Aug 06 00:51:25 dist-test-slave-45dx krb5kdc[2845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977484, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.188@KRBTEST.COM for kudu/127.15.147.188@KRBTEST.COM
I20260806 00:51:25.023619 3069 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } attempt: 3
Aug 06 00:51:25 dist-test-slave-45dx krb5kdc[2845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977484, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for kudu/127.15.147.188@KRBTEST.COM
I20260806 00:51:25.026587 3077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:25.017177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57682 (local address 127.15.147.189:34965)
0806 00:51:25.017305 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:25.017308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:25.017319 (+ 11us) server_negotiation.cc:407] Connection header received
0806 00:51:25.017420 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:25.017422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:25.017483 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:25.017563 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:25.018375 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.018843 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.019545 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.019775 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.019879 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:25.022529 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:25.022534 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:25.022538 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:25.022565 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:25.024524 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.025078 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.025082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.025083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.025167 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.025502 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.025505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.025507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.025730 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:25.025939 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:25.026317 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 00:51:25.026434 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:51:25.027170 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:25.018161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:43987 (local address 127.0.0.1:37178)
0806 00:51:25.018258 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:25.018270 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:25.018354 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:25.018627 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:25.018630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:25.018776 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:25.018831 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:25.018836 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.019514 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.019517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:25.019981 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:25.019989 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.020655 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.020660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:25.020746 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:25.021209 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:25.021228 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:25.022846 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:25.025610 (+ 2764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:25.025612 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:25.025614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:25.025840 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:25.026134 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:25.026137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:25.026139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:25.026228 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:25.026649 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:25.026653 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:25.026764 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 00:51:25.026875 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 00:51:25.026985 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"spinlock_wait_cycles":1408}
I20260806 00:51:25.027623 3185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:25.018307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37178 (local address 127.15.147.188:43987)
0806 00:51:25.018441 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:25.018444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:25.018454 (+ 10us) server_negotiation.cc:407] Connection header received
0806 00:51:25.018479 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:25.018481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:25.018537 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:25.018615 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:25.018938 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.019406 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.020091 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.020448 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.020551 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:25.023035 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:25.023039 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:25.023044 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:25.023066 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:25.025508 (+ 2442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.025955 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.025961 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.025966 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.026046 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.026318 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.026322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.026324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.026521 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:25.026612 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:25.027371 (+ 759us) server_negotiation.cc:299] Negotiation successful
0806 00:51:25.027487 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 00:51:25.030866 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:25.020546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:43987 (local address 127.0.0.1:37184)
0806 00:51:25.020707 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:25.020722 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:25.020790 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:25.021240 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:25.021243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:25.021398 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:25.021464 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:25.021469 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.022710 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.022713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:25.023474 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:25.023488 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.024382 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.024386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:25.024473 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:25.025053 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:25.025067 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:25.026551 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:25.029362 (+ 2811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:25.029365 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:25.029367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:25.029608 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:25.029890 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:25.029893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:25.029895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:25.029988 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:25.030344 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:25.030348 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:25.030448 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 00:51:25.030576 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 00:51:25.030717 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260806 00:51:25.030901 3187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:25.020586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37184 (local address 127.15.147.188:43987)
0806 00:51:25.021033 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:25.021044 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:25.021058 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:51:25.021092 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:25.021095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:25.021137 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:25.021224 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:25.021591 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.022528 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.023625 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.024088 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.024196 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:25.026705 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:25.026708 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:25.026711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:25.026748 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:25.029239 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.029707 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.029710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.029711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.029786 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.030084 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.030086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.030087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.030241 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:25.030349 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:25.030628 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 00:51:25.030727 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":106,"threads_started":1}
I20260806 00:51:25.031283 3175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb: Bootstrap starting.
I20260806 00:51:25.031397 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
Aug 06 00:51:25 dist-test-slave-45dx krb5kdc[2845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977484, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.189@KRBTEST.COM for kudu/127.15.147.188@KRBTEST.COM
I20260806 00:51:25.034060 2961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced: Bootstrap starting.
I20260806 00:51:25.034406 3175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:25.035637 3175 log.cc:824] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:25.036827 2961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:25.037758 2961 log.cc:824] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:25.038584 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:25.024035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:43987 (local address 127.0.0.1:37200)
0806 00:51:25.024150 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:25.024166 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:25.024238 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:25.025223 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:25.025228 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:25.025391 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:25.025514 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:25.025526 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.026709 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.026712 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:25.027845 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:25.027857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.032404 (+ 4547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.032410 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:25.032516 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:25.033007 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:25.033023 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:25.034648 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:25.036796 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:25.036799 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:25.036802 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:25.037084 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:25.037433 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:25.037437 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:25.037439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:25.037535 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:25.037986 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:25.037990 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:25.038101 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 00:51:25.038214 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 00:51:25.038398 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 00:51:25.039289 3188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:25.024209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37200 (local address 127.15.147.188:43987)
0806 00:51:25.024533 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:25.024545 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:25.024560 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:51:25.024615 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:25.024620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:25.025071 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:25.025651 (+ 580us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:25.025784 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.026587 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.027992 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.028408 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.028570 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:25.034786 (+ 6216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:25.034789 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:25.034792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:25.034816 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:25.036668 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.037218 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.037221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.037223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.037317 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.037647 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.037650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.037652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.037847 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:25.037966 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:25.039002 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0806 00:51:25.039124 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":141,"threads_started":1}
I20260806 00:51:25.039989 2961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced: No bootstrap required, opened a new log
I20260806 00:51:25.043185 3069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f: Bootstrap starting.
I20260806 00:51:25.045504 3187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:25.034311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37206 (local address 127.15.147.188:43987)
0806 00:51:25.034841 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:25.034845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:25.036068 (+ 1223us) server_negotiation.cc:407] Connection header received
0806 00:51:25.036249 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:25.036252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:25.036321 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:25.036375 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:25.037425 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.038146 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.039426 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.039735 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.039808 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:25.041723 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:25.041728 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:25.041732 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:25.041780 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:25.043581 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.044065 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.044068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.044070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.044154 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.044523 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.044544 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.044546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.044757 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:25.044863 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:25.045212 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 00:51:25.045327 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460}
I20260806 00:51:25.047384 3069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:25.047567 3175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb: No bootstrap required, opened a new log
I20260806 00:51:25.048445 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:25.038215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54568 (local address 127.15.147.190:46115)
0806 00:51:25.038333 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:25.038337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:25.038351 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:51:25.038389 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:25.038392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:25.038430 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:25.038501 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:25.039404 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.039945 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.041160 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.041491 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.041591 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:25.043714 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:25.043718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:25.043720 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:25.043745 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:25.045783 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.046359 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.046363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.046365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.046450 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.046746 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.046749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.046752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.046961 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:25.047081 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:25.048139 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0806 00:51:25.048266 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:51:25.048421 3069 log.cc:824] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:25.050545 3077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:25.035148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57684 (local address 127.15.147.189:34965)
0806 00:51:25.035267 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:25.035272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:25.041348 (+ 6076us) server_negotiation.cc:407] Connection header received
0806 00:51:25.041500 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:25.041503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:25.041553 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:25.041600 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:25.042481 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.043289 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.044468 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:25.044893 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:25.044983 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:25.046973 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:25.046977 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:25.046979 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:25.047011 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:25.048724 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.049212 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.049215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.049217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.049297 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:25.049591 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:25.049595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:25.049597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:25.049813 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:25.049943 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:25.050224 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 00:51:25.050370 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:51:25.052377 3175 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } }
I20260806 00:51:25.052600 3175 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:51:25.052672 3175 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01284c12eed949a497288848f75973bb, State: Initialized, Role: FOLLOWER
I20260806 00:51:25.053095 3175 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } }
I20260806 00:51:25.053611 3208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } } }
I20260806 00:51:25.053771 3208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:51:25.054005 3069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f: No bootstrap required, opened a new log
I20260806 00:51:25.054291 3175 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:51:25.056650 2961 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } }
I20260806 00:51:25.056939 2961 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:51:25.057042 2961 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4135655a334868b351d82525ad0ced, State: Initialized, Role: FOLLOWER
I20260806 00:51:25.057567 2961 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } }
I20260806 00:51:25.058975 2961 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:51:25.059015 3069 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } }
I20260806 00:51:25.059288 3069 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260806 00:51:25.059345 3224 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 00:51:25.059388 3069 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eea17f57aab4eafa7ee9fac5602ef8f, State: Initialized, Role: FOLLOWER
W20260806 00:51:25.059463 3224 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:51:25.059885 3069 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } }
I20260806 00:51:25.057771 3080 master_runner.cc:186] Error getting master registration for 127.15.147.190:46115: OK, instance_id { permanent_uuid: "fa4135655a334868b351d82525ad0ced" instance_seqno: 1785977484553957 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 00:51:25.061134 3080 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 00:51:25.061271 3246 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 00:51:25.061265 3069 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:51:25.064076 3227 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } } }
I20260806 00:51:25.064363 3227 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:51:25.064592 3238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } } }
I20260806 00:51:25.064750 3238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:51:25.068581 2864 master_runner.cc:263] No leader master found from master fa4135655a334868b351d82525ad0ced
I20260806 00:51:25.068647 2864 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 00:51:25.071771 2972 master_runner.cc:263] No leader master found from master 7eea17f57aab4eafa7ee9fac5602ef8f
I20260806 00:51:25.071841 2972 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 00:51:25.072789 3265 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:51:25.072875 3265 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:51:25.150810 3208 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:51:25.151005 3208 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } }
I20260806 00:51:25.152109 3208 leader_election.cc:288] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa4135655a334868b351d82525ad0ced (127.15.147.190:46115), 7eea17f57aab4eafa7ee9fac5602ef8f (127.15.147.189:34965)
I20260806 00:51:25.152585 2927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01284c12eed949a497288848f75973bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4135655a334868b351d82525ad0ced" is_pre_election: true
I20260806 00:51:25.152765 3033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01284c12eed949a497288848f75973bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" is_pre_election: true
I20260806 00:51:25.152899 2927 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01284c12eed949a497288848f75973bb in term 0.
I20260806 00:51:25.153028 3033 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01284c12eed949a497288848f75973bb in term 0.
I20260806 00:51:25.153302 3108 leader_election.cc:302] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01284c12eed949a497288848f75973bb, fa4135655a334868b351d82525ad0ced; no voters:
I20260806 00:51:25.153793 3208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:51:25.153893 3208 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:51:25.153944 3208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:25.155340 3208 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } }
I20260806 00:51:25.155685 3208 leader_election.cc:288] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [CANDIDATE]: Term 1 election: Requested vote from peers fa4135655a334868b351d82525ad0ced (127.15.147.190:46115), 7eea17f57aab4eafa7ee9fac5602ef8f (127.15.147.189:34965)
I20260806 00:51:25.156111 3033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01284c12eed949a497288848f75973bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f"
I20260806 00:51:25.156121 2927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01284c12eed949a497288848f75973bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa4135655a334868b351d82525ad0ced"
I20260806 00:51:25.156221 3033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:25.156263 2927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:25.157469 3033 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01284c12eed949a497288848f75973bb in term 1.
I20260806 00:51:25.157806 3109 leader_election.cc:302] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01284c12eed949a497288848f75973bb, 7eea17f57aab4eafa7ee9fac5602ef8f; no voters:
I20260806 00:51:25.158124 3208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:51:25.158227 2927 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01284c12eed949a497288848f75973bb in term 1.
I20260806 00:51:25.158396 3208 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [term 1 LEADER]: Becoming Leader. State: Replica: 01284c12eed949a497288848f75973bb, State: Running, Role: LEADER
I20260806 00:51:25.158731 3208 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } }
I20260806 00:51:25.160676 3208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01284c12eed949a497288848f75973bb. Latest consensus state: current_term: 1 leader_uuid: "01284c12eed949a497288848f75973bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } } }
I20260806 00:51:25.160846 3208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [sys.catalog]: This master's current role is: LEADER
I20260806 00:51:25.161374 3269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:51:25.162606 3269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:51:25.168087 3033 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f [term 1 FOLLOWER]: Refusing update from remote peer 01284c12eed949a497288848f75973bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:51:25.168445 2927 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced [term 1 FOLLOWER]: Refusing update from remote peer 01284c12eed949a497288848f75973bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:51:25.168946 3208 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:51:25.169173 3267 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:51:25.173399 3238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01284c12eed949a497288848f75973bb. Latest consensus state: current_term: 1 leader_uuid: "01284c12eed949a497288848f75973bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } } }
I20260806 00:51:25.173444 3227 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01284c12eed949a497288848f75973bb. Latest consensus state: current_term: 1 leader_uuid: "01284c12eed949a497288848f75973bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } } }
I20260806 00:51:25.173570 3227 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:51:25.173569 3238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:51:25.176316 3208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "01284c12eed949a497288848f75973bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } } }
I20260806 00:51:25.176437 3208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [sys.catalog]: This master's current role is: LEADER
I20260806 00:51:25.176544 3208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "01284c12eed949a497288848f75973bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } } }
I20260806 00:51:25.176607 3208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb [sys.catalog]: This master's current role is: LEADER
I20260806 00:51:25.176681 3227 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "01284c12eed949a497288848f75973bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } } }
I20260806 00:51:25.176782 3227 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:51:25.178954 3238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "01284c12eed949a497288848f75973bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa4135655a334868b351d82525ad0ced" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 46115 } } peers { permanent_uuid: "7eea17f57aab4eafa7ee9fac5602ef8f" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34965 } } peers { permanent_uuid: "01284c12eed949a497288848f75973bb" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 43987 } } }
I20260806 00:51:25.179064 3238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:51:25.180445 3272 mvcc.cc:205] Tried to move back new op lower bound from 7315363779241684992 to 7315363779213582336. Current Snapshot: MvccSnapshot[applied={T|T < 7315363779241684992}]
I20260806 00:51:25.184369 3271 mvcc.cc:205] Tried to move back new op lower bound from 7315363779241684992 to 7315363779213582336. Current Snapshot: MvccSnapshot[applied={T|T < 7315363779241684992}]
I20260806 00:51:25.184847 3269 catalog_manager.cc:1347] Generated new cluster ID: 6bae53699d71461d99cc592a08739ea7
I20260806 00:51:25.184918 3269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:51:25.210377 3269 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:51:25.211614 3269 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:51:25.226389 3269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01284c12eed949a497288848f75973bb: Generated new TSK 0
I20260806 00:51:25.227883 3269 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:51:26.064777 3246 catalog_manager.cc:1259] Loaded cluster ID: 6bae53699d71461d99cc592a08739ea7
I20260806 00:51:26.064890 3246 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced: loading cluster ID for follower catalog manager: success
I20260806 00:51:26.066597 3246 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced: acquiring CA information for follower catalog manager: success
I20260806 00:51:26.067838 3246 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fa4135655a334868b351d82525ad0ced: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:51:26.073958 3265 catalog_manager.cc:1259] Loaded cluster ID: 6bae53699d71461d99cc592a08739ea7
I20260806 00:51:26.074052 3265 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f: loading cluster ID for follower catalog manager: success
I20260806 00:51:26.075591 3265 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f: acquiring CA information for follower catalog manager: success
I20260806 00:51:26.076751 3265 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7eea17f57aab4eafa7ee9fac5602ef8f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:51:26.289716 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 2864
I20260806 00:51:26.303344 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 2972
I20260806 00:51:26.315140 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 3080
2026-08-06T00:51:26Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3313 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:51:26 dist-test-slave-45dx krb5kdc[3289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:51:26 dist-test-slave-45dx krb5kdc[3289](info): set up 2 sockets
Aug 06 00:51:26 dist-test-slave-45dx krb5kdc[3289](info): commencing operation
krb5kdc: starting...
W20260806 00:51:27.521687 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:51:27 dist-test-slave-45dx krb5kdc[3289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:51:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:51:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:51:27.653527 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:33381
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:46473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:33381
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:51:27.764467 3307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:27.764757 3307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:27.764827 3307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:27.768857 3307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:51:27.768932 3307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:27.768965 3307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:27.769027 3307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:51:27.769068 3307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:51:27.774173 3307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:46473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:33381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:33381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:27.775177 3307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:27.776335 3307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:27.784878 3318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:27.784906 3322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:27.784978 3307 server_base.cc:1034] running on GCE node
W20260806 00:51:27.784889 3317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:27.785812 3307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:27.786864 3307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:27.788059 3307 hybrid_clock.cc:648] HybridClock initialized: now 1785977487788045 us; error 32 us; skew 500 ppm
Aug 06 00:51:27 dist-test-slave-45dx krb5kdc[3289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977487, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:27.790890 3307 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:51:27.791759 3307 webserver.cc:460] Webserver started at http://127.15.147.190:39195/ using document root <none> and password file <none>
I20260806 00:51:27.792294 3307 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:27.792445 3307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:27.792680 3307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:51:27.794693 3307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "13e852c9df594bc4aa812927f4fc8c27"
format_stamp: "Formatted at 2026-08-06 00:51:27 on dist-test-slave-45dx"
server_key: "d9bb89df1796b775027cc2cb2044fd62"
server_key_iv: "cac3c3955a634f913c669daf5ae8d920"
server_key_version: "encryptionkey@0"
I20260806 00:51:27.795249 3307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "13e852c9df594bc4aa812927f4fc8c27"
format_stamp: "Formatted at 2026-08-06 00:51:27 on dist-test-slave-45dx"
server_key: "d9bb89df1796b775027cc2cb2044fd62"
server_key_iv: "cac3c3955a634f913c669daf5ae8d920"
server_key_version: "encryptionkey@0"
I20260806 00:51:27.798911 3307 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:51:27.801375 3331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:27.802584 3307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:51:27.802729 3307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "13e852c9df594bc4aa812927f4fc8c27"
format_stamp: "Formatted at 2026-08-06 00:51:27 on dist-test-slave-45dx"
server_key: "d9bb89df1796b775027cc2cb2044fd62"
server_key_iv: "cac3c3955a634f913c669daf5ae8d920"
server_key_version: "encryptionkey@0"
I20260806 00:51:27.802845 3307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:27.812359 3307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:27.815217 3307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:27.815428 3307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:27.823338 3307 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:33381
I20260806 00:51:27.823357 3403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:33381 every 8 connection(s)
I20260806 00:51:27.824295 3307 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:51:27.827065 3404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:27.829775 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 3307
I20260806 00:51:27.829873 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:51:27.830168 15950 external_mini_cluster.cc:1428] Setting key f391a3f53dbc9d5f2856e8e10a6ed748
I20260806 00:51:27.833242 3404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27: Bootstrap starting.
I20260806 00:51:27.835990 3404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:27.836861 3404 log.cc:824] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:27.838912 3404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27: No bootstrap required, opened a new log
Aug 06 00:51:27 dist-test-slave-45dx krb5kdc[3289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:51:27.842978 3404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13e852c9df594bc4aa812927f4fc8c27" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33381 } }
I20260806 00:51:27.843262 3404 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:51:27.843336 3404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13e852c9df594bc4aa812927f4fc8c27, State: Initialized, Role: FOLLOWER
I20260806 00:51:27.843816 3404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13e852c9df594bc4aa812927f4fc8c27" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33381 } }
I20260806 00:51:27.843981 3404 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:51:27.844048 3404 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:51:27.844148 3404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:27.845258 3404 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13e852c9df594bc4aa812927f4fc8c27" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33381 } }
I20260806 00:51:27.845628 3404 leader_election.cc:302] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13e852c9df594bc4aa812927f4fc8c27; no voters:
I20260806 00:51:27.845911 3409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:27.831551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56320 (local address 127.15.147.190:33381)
0806 00:51:27.831979 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:27.831998 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:27.832041 (+ 43us) server_negotiation.cc:407] Connection header received
0806 00:51:27.832873 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:27.832897 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:27.833295 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:27.833659 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:27.834372 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:27.835865 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:27.836804 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:27.837456 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:27.837574 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:27.840008 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:27.840014 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:27.840029 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:27.840059 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:27.842649 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:27.843075 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:27.843080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:27.843086 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:27.843225 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:27.843588 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:27.843591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:27.843593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:27.844083 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:27.844322 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:27.844597 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 00:51:27.844878 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":106,"threads_started":1}
I20260806 00:51:27.845986 3404 leader_election.cc:288] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:51:27.846102 3411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:51:27.846330 3411 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27 [term 1 LEADER]: Becoming Leader. State: Replica: 13e852c9df594bc4aa812927f4fc8c27, State: Running, Role: LEADER
I20260806 00:51:27.846696 3411 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13e852c9df594bc4aa812927f4fc8c27" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33381 } }
I20260806 00:51:27.847203 3404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:51:27.848492 3413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13e852c9df594bc4aa812927f4fc8c27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13e852c9df594bc4aa812927f4fc8c27" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33381 } } }
I20260806 00:51:27.848623 3413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27 [sys.catalog]: This master's current role is: LEADER
I20260806 00:51:27.848896 3414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13e852c9df594bc4aa812927f4fc8c27. Latest consensus state: current_term: 1 leader_uuid: "13e852c9df594bc4aa812927f4fc8c27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13e852c9df594bc4aa812927f4fc8c27" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33381 } } }
I20260806 00:51:27.848990 3414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27 [sys.catalog]: This master's current role is: LEADER
I20260806 00:51:27.849100 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 00:51:27.852247 3433 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:51:27.852352 3433 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:51:27.852423 3423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:51:27.853322 3423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:51:27.858325 3423 catalog_manager.cc:1347] Generated new cluster ID: 526ad05c7781448dae383f2e30749b1d
I20260806 00:51:27.858392 3423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:51:27.868953 3423 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:51:27.869843 3423 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:51:27.875473 3423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13e852c9df594bc4aa812927f4fc8c27: Generated new TSK 0
I20260806 00:51:27.876200 3423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:51:27.905253 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 3307
2026-08-06T00:51:27Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1585 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:51:27 dist-test-slave-45dx krb5kdc[3445](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:51:27 dist-test-slave-45dx krb5kdc[3445](info): set up 2 sockets
Aug 06 00:51:27 dist-test-slave-45dx krb5kdc[3445](info): commencing operation
krb5kdc: starting...
W20260806 00:51:29.084107 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:51:29 dist-test-slave-45dx krb5kdc[3445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:51:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:51:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:51:29.207149 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:40217
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:42903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:40217
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:51:29.316171 3459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:29.316471 3459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:29.316548 3459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:29.320196 3459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:51:29.320277 3459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:29.320328 3459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:29.320370 3459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:51:29.320425 3459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:51:29.325349 3459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:42903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:40217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:40217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:29.326272 3459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:29.327245 3459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:29.335680 3468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:29.335690 3465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:29.335693 3464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:29.335727 3459 server_base.cc:1034] running on GCE node
I20260806 00:51:29.336603 3459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:29.337630 3459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:29.338817 3459 hybrid_clock.cc:648] HybridClock initialized: now 1785977489338787 us; error 43 us; skew 500 ppm
Aug 06 00:51:29 dist-test-slave-45dx krb5kdc[3445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977489, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:29.345501 3459 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:51:29.346374 3459 webserver.cc:460] Webserver started at http://127.15.147.190:38655/ using document root <none> and password file <none>
I20260806 00:51:29.346948 3459 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:29.347045 3459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:29.347283 3459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:51:29.349390 3459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "5d7cf4b210e843a9b43f587708ec8014"
format_stamp: "Formatted at 2026-08-06 00:51:29 on dist-test-slave-45dx"
server_key: "6b7f4f6aed4968d1a6a5caa1ff81f75c"
server_key_iv: "dcb8346baad17a609f930fea2e42e971"
server_key_version: "encryptionkey@0"
I20260806 00:51:29.349967 3459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "5d7cf4b210e843a9b43f587708ec8014"
format_stamp: "Formatted at 2026-08-06 00:51:29 on dist-test-slave-45dx"
server_key: "6b7f4f6aed4968d1a6a5caa1ff81f75c"
server_key_iv: "dcb8346baad17a609f930fea2e42e971"
server_key_version: "encryptionkey@0"
I20260806 00:51:29.353648 3459 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:51:29.356038 3480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:29.357192 3459 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:51:29.357349 3459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "5d7cf4b210e843a9b43f587708ec8014"
format_stamp: "Formatted at 2026-08-06 00:51:29 on dist-test-slave-45dx"
server_key: "6b7f4f6aed4968d1a6a5caa1ff81f75c"
server_key_iv: "dcb8346baad17a609f930fea2e42e971"
server_key_version: "encryptionkey@0"
I20260806 00:51:29.357481 3459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:29.367102 3459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:29.369809 3459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:29.370005 3459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:29.376859 3459 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:40217
I20260806 00:51:29.377132 3552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:40217 every 8 connection(s)
I20260806 00:51:29.377799 3459 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:51:29.380638 3554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:29.383294 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 3459
I20260806 00:51:29.383401 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:51:29.383704 15950 external_mini_cluster.cc:1428] Setting key 41556540c76342fb8c8fe08bd5abdd76
I20260806 00:51:29.386394 3554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014: Bootstrap starting.
I20260806 00:51:29.389058 3554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:29.389919 3554 log.cc:824] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:29.392170 3554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014: No bootstrap required, opened a new log
Aug 06 00:51:29 dist-test-slave-45dx krb5kdc[3445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:51:29.395051 3554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7cf4b210e843a9b43f587708ec8014" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40217 } }
I20260806 00:51:29.395254 3554 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:51:29.395323 3554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d7cf4b210e843a9b43f587708ec8014, State: Initialized, Role: FOLLOWER
I20260806 00:51:29.395808 3554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7cf4b210e843a9b43f587708ec8014" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40217 } }
I20260806 00:51:29.395963 3554 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:51:29.396057 3554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:51:29.396165 3554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:29.397061 3554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7cf4b210e843a9b43f587708ec8014" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40217 } }
I20260806 00:51:29.397392 3554 leader_election.cc:302] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d7cf4b210e843a9b43f587708ec8014; no voters:
I20260806 00:51:29.397742 3554 leader_election.cc:288] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:51:29.397818 3561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:51:29.398058 3561 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 1 LEADER]: Becoming Leader. State: Replica: 5d7cf4b210e843a9b43f587708ec8014, State: Running, Role: LEADER
I20260806 00:51:29.398411 3561 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7cf4b210e843a9b43f587708ec8014" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40217 } }
I20260806 00:51:29.398716 3559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:29.385027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33208 (local address 127.15.147.190:40217)
0806 00:51:29.385435 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:29.385450 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:29.385483 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:51:29.386068 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:29.386084 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:29.386387 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:29.386690 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:29.387587 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:29.388604 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:29.389751 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:29.390260 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:29.390448 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:29.393217 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:29.393223 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:29.393234 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:29.393277 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:29.395638 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:29.396094 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:29.396107 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:29.396111 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:29.396276 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:29.396606 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:29.396609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:29.396611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:29.396989 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:29.397182 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:29.397471 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 00:51:29.397774 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":130,"threads_started":1}
I20260806 00:51:29.399003 3554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:51:29.400159 3563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d7cf4b210e843a9b43f587708ec8014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7cf4b210e843a9b43f587708ec8014" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40217 } } }
I20260806 00:51:29.400285 3563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [sys.catalog]: This master's current role is: LEADER
I20260806 00:51:29.400629 3564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d7cf4b210e843a9b43f587708ec8014. Latest consensus state: current_term: 1 leader_uuid: "5d7cf4b210e843a9b43f587708ec8014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7cf4b210e843a9b43f587708ec8014" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40217 } } }
I20260806 00:51:29.401376 3572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:51:29.401536 3564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [sys.catalog]: This master's current role is: LEADER
I20260806 00:51:29.404534 3572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:51:29.410075 3572 catalog_manager.cc:1347] Generated new cluster ID: fa26c7676ceb45069c6b3eb19d06ce20
I20260806 00:51:29.410142 3572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:51:29.430996 3572 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:51:29.432209 3572 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:51:29.439323 3572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014: Generated new TSK 0
I20260806 00:51:29.439982 3572 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
I20260806 00:51:29.477284 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:40217
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:42903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:51:29.587030 3587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:29.587281 3587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:29.587352 3587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:29.591244 3587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:29.591327 3587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:29.591416 3587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:51:29.596496 3587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:42903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:40217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:29.597478 3587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:29.598472 3587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:29.607925 3599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:29.608000 3596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:29.607918 3595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:29.608075 3587 server_base.cc:1034] running on GCE node
I20260806 00:51:29.608595 3587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:29.609176 3587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:29.610330 3587 hybrid_clock.cc:648] HybridClock initialized: now 1785977489610311 us; error 32 us; skew 500 ppm
Aug 06 00:51:29 dist-test-slave-45dx krb5kdc[3445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977489, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:29.612999 3587 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:51:29.613893 3587 webserver.cc:460] Webserver started at http://127.15.147.129:37443/ using document root <none> and password file <none>
I20260806 00:51:29.614446 3587 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:29.614507 3587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:29.614755 3587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:51:29.616727 3587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "cd7245054dd14610861e2ba0e7821a48"
format_stamp: "Formatted at 2026-08-06 00:51:29 on dist-test-slave-45dx"
server_key: "2503862d01c992cb028f7db7f939a878"
server_key_iv: "8cf9a9a34db740548bf63921e1552486"
server_key_version: "encryptionkey@0"
I20260806 00:51:29.617311 3587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "cd7245054dd14610861e2ba0e7821a48"
format_stamp: "Formatted at 2026-08-06 00:51:29 on dist-test-slave-45dx"
server_key: "2503862d01c992cb028f7db7f939a878"
server_key_iv: "8cf9a9a34db740548bf63921e1552486"
server_key_version: "encryptionkey@0"
I20260806 00:51:29.620864 3587 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:51:29.623114 3611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:29.624281 3587 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:51:29.624490 3587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "cd7245054dd14610861e2ba0e7821a48"
format_stamp: "Formatted at 2026-08-06 00:51:29 on dist-test-slave-45dx"
server_key: "2503862d01c992cb028f7db7f939a878"
server_key_iv: "8cf9a9a34db740548bf63921e1552486"
server_key_version: "encryptionkey@0"
I20260806 00:51:29.624598 3587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:29.649534 3587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:29.652361 3587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:29.652559 3587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:29.653136 3587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:51:29.653959 3587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:51:29.654031 3587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:29.654074 3587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:51:29.654096 3587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:29.663115 3587 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:42081
I20260806 00:51:29.663156 3773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:42081 every 8 connection(s)
I20260806 00:51:29.663959 3587 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:51:29.664369 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 3587
I20260806 00:51:29.664494 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:51:29.664856 15950 external_mini_cluster.cc:1428] Setting key 0f29ac072be3b8e128a5579dd3138252
Aug 06 00:51:29 dist-test-slave-45dx krb5kdc[3445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977489, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.147.130@KRBTEST.COM; defaulting to no policy
I20260806 00:51:29.678869 3559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:29.665802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:39295 (local address 127.15.147.190:40217)
0806 00:51:29.665938 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:29.665942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:29.666954 (+ 1012us) server_negotiation.cc:407] Connection header received
0806 00:51:29.669091 (+ 2137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:29.669094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:29.669148 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:29.669255 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:29.670357 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:29.671166 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:29.672254 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:29.672544 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:29.672661 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:29.675117 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:29.675121 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:29.675123 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:29.675156 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:29.676750 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:29.677282 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:29.677286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:29.677288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:29.677406 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:29.677767 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:29.677771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:29.677773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:29.677968 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:29.678077 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:29.678565 (+ 488us) server_negotiation.cc:299] Negotiation successful
0806 00:51:29.678695 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.130@KRBTEST.COM" created.
I20260806 00:51:29.679822 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:29.666257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:39295)
0806 00:51:29.666798 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:29.666861 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:29.668888 (+ 2027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:29.669430 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:29.669436 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:29.669776 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:29.670215 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:29.670229 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:29.671301 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:29.671304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:29.672126 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:29.672136 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:29.672700 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:29.672704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:29.672870 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:29.673502 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:29.673517 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:29.674955 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:29.676887 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:29.676893 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:29.676907 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:29.677171 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:29.677515 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:29.677519 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:29.677521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:29.677666 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:29.678077 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:29.678085 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:29.678320 (+ 235us) client_negotiation.cc:769] Sending connection context
0806 00:51:29.678522 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 00:51:29.678917 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":132,"threads_started":1}
I20260806 00:51:29.681182 3778 heartbeater.cc:347] Connected to a master server at 127.15.147.190:40217
I20260806 00:51:29.681466 3778 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:29.682096 3778 heartbeater.cc:511] Master 127.15.147.190:40217 requested a full tablet report, sending...
I20260806 00:51:29.683655 3504 ts_manager.cc:194] Registered new tserver with Master: cd7245054dd14610861e2ba0e7821a48 (127.15.147.129:42081)
I20260806 00:51:29.684832 3504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:39295
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
I20260806 00:51:29.696237 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:40217
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:42903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:51:29.807104 3784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:29.807323 3784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:29.807363 3784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:29.811131 3784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:29.811174 3784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:29.811228 3784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:51:29.816079 3784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:42903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:40217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:29.817052 3784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:29.818020 3784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:29.827854 3790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:29.827903 3793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:29.827893 3791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:29.827965 3784 server_base.cc:1034] running on GCE node
I20260806 00:51:29.828543 3784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:29.829169 3784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:29.830355 3784 hybrid_clock.cc:648] HybridClock initialized: now 1785977489830330 us; error 46 us; skew 500 ppm
Aug 06 00:51:29 dist-test-slave-45dx krb5kdc[3445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977489, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:29.833213 3784 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:51:29.834085 3784 webserver.cc:460] Webserver started at http://127.15.147.130:44453/ using document root <none> and password file <none>
I20260806 00:51:29.834637 3784 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:29.834731 3784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:29.834960 3784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:51:29.836984 3784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "ee5e9fbd481c4102aa9bcde8485eb84f"
format_stamp: "Formatted at 2026-08-06 00:51:29 on dist-test-slave-45dx"
server_key: "6b236eccd7116b97657b0ad6e47dfc84"
server_key_iv: "2f4e0301cc0220f33259a79f9fad3750"
server_key_version: "encryptionkey@0"
I20260806 00:51:29.837568 3784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "ee5e9fbd481c4102aa9bcde8485eb84f"
format_stamp: "Formatted at 2026-08-06 00:51:29 on dist-test-slave-45dx"
server_key: "6b236eccd7116b97657b0ad6e47dfc84"
server_key_iv: "2f4e0301cc0220f33259a79f9fad3750"
server_key_version: "encryptionkey@0"
I20260806 00:51:29.841106 3784 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:51:29.843461 3802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:29.844662 3784 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260806 00:51:29.844792 3784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "ee5e9fbd481c4102aa9bcde8485eb84f"
format_stamp: "Formatted at 2026-08-06 00:51:29 on dist-test-slave-45dx"
server_key: "6b236eccd7116b97657b0ad6e47dfc84"
server_key_iv: "2f4e0301cc0220f33259a79f9fad3750"
server_key_version: "encryptionkey@0"
I20260806 00:51:29.844899 3784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:29.867403 3784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:29.870775 3784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:29.870983 3784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:29.871593 3784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:51:29.872486 3784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:51:29.872552 3784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:29.872609 3784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:51:29.872656 3784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:29.882444 3784 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:40387
I20260806 00:51:29.882558 3957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:40387 every 8 connection(s)
I20260806 00:51:29.883440 3784 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:51:29.893098 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 3784
I20260806 00:51:29.893237 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
Aug 06 00:51:29 dist-test-slave-45dx krb5kdc[3445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977489, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:51:29.893565 15950 external_mini_cluster.cc:1428] Setting key 410944e6fd3b41bd4f5120fcce57d6ae
I20260806 00:51:29.897751 3559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:29.885429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:37901 (local address 127.15.147.190:40217)
0806 00:51:29.885578 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:29.885582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:29.886438 (+ 856us) server_negotiation.cc:407] Connection header received
0806 00:51:29.887895 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:29.887899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:29.887957 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:29.888083 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:29.889331 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:29.889857 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:29.891053 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:29.891269 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:29.891378 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:29.894073 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:29.894077 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:29.894080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:29.894123 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:29.895665 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:29.896179 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:29.896183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:29.896184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:29.896266 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:29.896650 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:29.896654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:29.896656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:29.896846 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:29.896952 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:29.897441 (+ 489us) server_negotiation.cc:299] Negotiation successful
0806 00:51:29.897575 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:51:29.899597 3960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:29.885809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:37901)
0806 00:51:29.886289 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:29.886334 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:29.887685 (+ 1351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:29.888268 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:29.888275 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:29.888725 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:29.889182 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:29.889195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:29.889990 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:29.889994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:29.890920 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:29.890930 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:29.891422 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:29.891427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:29.891597 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:29.892196 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:29.892212 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:29.893910 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:29.895786 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:29.895792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:29.895805 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:29.896062 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:29.896404 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:29.896407 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:29.896409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:29.896551 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:29.896952 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:29.896960 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:29.897190 (+ 230us) client_negotiation.cc:769] Sending connection context
0806 00:51:29.897388 (+ 198us) client_negotiation.cc:240] Negotiation successful
0806 00:51:29.897718 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":126,"threads_started":1}
I20260806 00:51:29.901032 3958 heartbeater.cc:347] Connected to a master server at 127.15.147.190:40217
I20260806 00:51:29.901304 3958 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:29.901877 3958 heartbeater.cc:511] Master 127.15.147.190:40217 requested a full tablet report, sending...
I20260806 00:51:29.902978 3504 ts_manager.cc:194] Registered new tserver with Master: ee5e9fbd481c4102aa9bcde8485eb84f (127.15.147.130:40387)
I20260806 00:51:29.903795 3504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:37901
WARNING: no policy specified for kudu/127.15.147.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
I20260806 00:51:29.927249 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:40217
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:42903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:51:30.039058 3963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:30.039307 3963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:30.039358 3963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:30.043237 3963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:30.043284 3963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:30.043344 3963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:51:30.048364 3963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:42903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:40217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:30.049463 3963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:30.050462 3963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:30.060032 3971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:30.060118 3970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:30.060036 3974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:30.060482 3963 server_base.cc:1034] running on GCE node
I20260806 00:51:30.060864 3963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:30.061517 3963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:30.062716 3963 hybrid_clock.cc:648] HybridClock initialized: now 1785977490062692 us; error 41 us; skew 500 ppm
Aug 06 00:51:30 dist-test-slave-45dx krb5kdc[3445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977490, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:30.065500 3963 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:51:30.066299 3963 webserver.cc:460] Webserver started at http://127.15.147.131:45663/ using document root <none> and password file <none>
I20260806 00:51:30.066829 3963 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:30.066920 3963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:30.067158 3963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:51:30.069231 3963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "4c1e7f983def48678fd856d6655b41f9"
format_stamp: "Formatted at 2026-08-06 00:51:30 on dist-test-slave-45dx"
server_key: "f7fe7360381dccaf9aafe5e7eee85699"
server_key_iv: "6665caff47f966b534c70edadebaf320"
server_key_version: "encryptionkey@0"
I20260806 00:51:30.069818 3963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "4c1e7f983def48678fd856d6655b41f9"
format_stamp: "Formatted at 2026-08-06 00:51:30 on dist-test-slave-45dx"
server_key: "f7fe7360381dccaf9aafe5e7eee85699"
server_key_iv: "6665caff47f966b534c70edadebaf320"
server_key_version: "encryptionkey@0"
I20260806 00:51:30.073482 3963 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:51:30.075752 3983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:30.076970 3963 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:51:30.077116 3963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "4c1e7f983def48678fd856d6655b41f9"
format_stamp: "Formatted at 2026-08-06 00:51:30 on dist-test-slave-45dx"
server_key: "f7fe7360381dccaf9aafe5e7eee85699"
server_key_iv: "6665caff47f966b534c70edadebaf320"
server_key_version: "encryptionkey@0"
I20260806 00:51:30.077226 3963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:30.097622 3963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:30.100672 3963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:30.100860 3963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:30.101496 3963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:51:30.102628 3963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:51:30.102699 3963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:30.102761 3963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:51:30.102793 3963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:30.111846 3963 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:38769
I20260806 00:51:30.111886 4147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:38769 every 8 connection(s)
I20260806 00:51:30.112771 3963 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:51:30.114775 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 3963
I20260806 00:51:30.114892 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:51:30.115177 15950 external_mini_cluster.cc:1428] Setting key ddd4594a1237e685b085cfcdc4c27cb3
Aug 06 00:51:30 dist-test-slave-45dx krb5kdc[3445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977490, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:51:30.125635 3559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.114510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:59711 (local address 127.15.147.190:40217)
0806 00:51:30.114653 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:30.114658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:30.115592 (+ 934us) server_negotiation.cc:407] Connection header received
0806 00:51:30.117042 (+ 1450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:30.117045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:30.117101 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:30.117188 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:30.118291 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.118804 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.119877 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.120084 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.120161 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.122626 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:30.122629 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:30.122631 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:30.122653 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:30.123883 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:30.124396 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:30.124398 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:30.124400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:30.124475 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:30.124770 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:30.124772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:30.124773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:30.124934 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:30.125009 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:30.125429 (+ 420us) server_negotiation.cc:299] Negotiation successful
0806 00:51:30.125520 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:51:30.126396 4150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.114954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:59711)
0806 00:51:30.115418 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:30.115488 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:30.116845 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:30.117378 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:30.117385 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:30.117723 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:30.118167 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:30.118180 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.118906 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.118908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:30.119772 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:30.119781 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.120205 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.120208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:30.120416 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.121029 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:30.121044 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:30.122471 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:30.123984 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:30.123988 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:30.123998 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:30.124234 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:30.124556 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:30.124559 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:30.124560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:30.124688 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:30.125026 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:30.125031 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:30.125231 (+ 200us) client_negotiation.cc:769] Sending connection context
0806 00:51:30.125382 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 00:51:30.125602 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":127,"threads_started":1}
I20260806 00:51:30.127506 4148 heartbeater.cc:347] Connected to a master server at 127.15.147.190:40217
I20260806 00:51:30.127727 4148 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:30.128288 4148 heartbeater.cc:511] Master 127.15.147.190:40217 requested a full tablet report, sending...
I20260806 00:51:30.129403 3504 ts_manager.cc:194] Registered new tserver with Master: 4c1e7f983def48678fd856d6655b41f9 (127.15.147.131:38769)
I20260806 00:51:30.129987 3504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:59711
I20260806 00:51:30.141216 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:51:30.150193 3559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.143254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33210 (local address 127.15.147.190:40217)
0806 00:51:30.143381 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:30.143385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:30.143559 (+ 174us) server_negotiation.cc:407] Connection header received
0806 00:51:30.143644 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:30.143646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:30.143685 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:30.143750 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:30.144486 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.144957 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.145746 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.145947 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.146021 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.147305 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:30.147307 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:30.147309 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:30.147357 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:30.148834 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:30.149252 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:30.149255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:30.149256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:30.149330 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:30.149547 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:30.149549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:30.149550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:30.149735 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:30.149832 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:30.149989 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 00:51:30.150085 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:51:30.152992 3504 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33210:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:51:30.155978 3504 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:51:30.170151 4161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.163994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:38769 (local address 127.0.0.1:43166)
0806 00:51:30.164371 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:30.164424 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:30.164552 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:30.165623 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:30.165629 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:30.165652 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:30.165783 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:30.165790 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.167242 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.167248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:30.168552 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:30.168562 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.169598 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.169602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:30.169706 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.169750 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 00:51:30.169870 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 00:51:30.169986 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":53,"thread_start_us":101,"threads_started":1}
I20260806 00:51:30.170194 4162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.164179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43166 (local address 127.15.147.131:38769)
0806 00:51:30.165117 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:30.165130 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:30.165158 (+ 28us) server_negotiation.cc:407] Connection header received
0806 00:51:30.165241 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:30.165246 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:30.165457 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:30.165633 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:30.165918 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.167090 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.168677 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.169450 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.169592 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.169720 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:30.169932 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 00:51:30.170034 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":830,"thread_start_us":139,"threads_started":1}
I20260806 00:51:30.170552 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.164430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:40387 (local address 127.0.0.1:59482)
0806 00:51:30.165463 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:30.165477 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:30.165584 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:30.166327 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:30.166331 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:30.166353 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:30.166433 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:30.166439 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.167778 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.167781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:30.168823 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:30.168834 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.170155 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.170158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:30.170258 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.170280 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:51:30.170349 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 00:51:30.170418 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":974,"thread_start_us":111,"threads_started":1}
I20260806 00:51:30.170748 4164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.164744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:42081 (local address 127.0.0.1:38212)
0806 00:51:30.165640 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:30.165654 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:30.165737 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:30.166125 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:30.166128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:30.166145 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:30.166242 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:30.166249 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.167340 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.167344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:30.169117 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:30.169125 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.170300 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.170304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:30.170417 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.170451 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 00:51:30.170522 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 00:51:30.170595 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"mutex_wait_us":533,"thread_start_us":105,"threads_started":1}
I20260806 00:51:30.171036 4169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.164877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59482 (local address 127.15.147.130:40387)
0806 00:51:30.165325 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:30.165345 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:30.165938 (+ 593us) server_negotiation.cc:407] Connection header received
0806 00:51:30.166022 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:30.166027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:30.166206 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:30.166359 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:30.166529 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.167627 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.168956 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.169819 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.169959 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.170087 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:30.170774 (+ 687us) server_negotiation.cc:299] Negotiation successful
0806 00:51:30.170870 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":155,"threads_started":1}
I20260806 00:51:30.171265 4166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.164921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38212 (local address 127.15.147.129:42081)
0806 00:51:30.165491 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:30.165509 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:30.165728 (+ 219us) server_negotiation.cc:407] Connection header received
0806 00:51:30.165848 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:30.165852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:30.165994 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:30.166146 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:30.166373 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.167163 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.169229 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.169988 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.170103 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.170225 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:30.170979 (+ 754us) server_negotiation.cc:299] Negotiation successful
0806 00:51:30.171066 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":99,"threads_started":1}
I20260806 00:51:30.173372 3867 tablet_service.cc:1467] Processing CreateTablet for tablet b09bf4f0c1b147c0897d960285a95221 (DEFAULT_TABLE table=test-table [id=02bb1f01d93041838b47d899a3c579fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:51:30.173413 4051 tablet_service.cc:1467] Processing CreateTablet for tablet b09bf4f0c1b147c0897d960285a95221 (DEFAULT_TABLE table=test-table [id=02bb1f01d93041838b47d899a3c579fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:51:30.173693 3682 tablet_service.cc:1467] Processing CreateTablet for tablet b09bf4f0c1b147c0897d960285a95221 (DEFAULT_TABLE table=test-table [id=02bb1f01d93041838b47d899a3c579fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:51:30.174269 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b09bf4f0c1b147c0897d960285a95221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:30.174461 4051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b09bf4f0c1b147c0897d960285a95221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:30.174515 3682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b09bf4f0c1b147c0897d960285a95221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:30.180820 4170 tablet_bootstrap.cc:492] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f: Bootstrap starting.
I20260806 00:51:30.182116 4172 tablet_bootstrap.cc:492] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48: Bootstrap starting.
I20260806 00:51:30.182641 4173 tablet_bootstrap.cc:492] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9: Bootstrap starting.
I20260806 00:51:30.183849 4170 tablet_bootstrap.cc:654] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:30.183998 4172 tablet_bootstrap.cc:654] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:30.184794 4172 log.cc:824] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:30.184818 4170 log.cc:824] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:30.184881 4173 tablet_bootstrap.cc:654] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:30.186108 4173 log.cc:824] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:30.186785 4172 tablet_bootstrap.cc:492] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48: No bootstrap required, opened a new log
I20260806 00:51:30.186800 4170 tablet_bootstrap.cc:492] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f: No bootstrap required, opened a new log
I20260806 00:51:30.186986 4172 ts_tablet_manager.cc:1397] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 00:51:30.186992 4170 ts_tablet_manager.cc:1397] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:51:30.188061 4173 tablet_bootstrap.cc:492] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9: No bootstrap required, opened a new log
I20260806 00:51:30.188235 4173 ts_tablet_manager.cc:1397] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:51:30.190129 4172 raft_consensus.cc:348] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } } peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } }
I20260806 00:51:30.190362 4172 raft_consensus.cc:374] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:51:30.190416 4172 raft_consensus.cc:729] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd7245054dd14610861e2ba0e7821a48, State: Initialized, Role: FOLLOWER
I20260806 00:51:30.190737 4170 raft_consensus.cc:348] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } } peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } }
I20260806 00:51:30.190975 4170 raft_consensus.cc:374] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:51:30.190893 4172 consensus_queue.cc:260] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } } peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } }
I20260806 00:51:30.191022 4170 raft_consensus.cc:729] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5e9fbd481c4102aa9bcde8485eb84f, State: Initialized, Role: FOLLOWER
I20260806 00:51:30.191437 4170 consensus_queue.cc:260] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } } peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } }
I20260806 00:51:30.191768 3778 heartbeater.cc:503] Master 127.15.147.190:40217 was elected leader, sending a full tablet report...
I20260806 00:51:30.192106 3958 heartbeater.cc:503] Master 127.15.147.190:40217 was elected leader, sending a full tablet report...
I20260806 00:51:30.192361 4170 ts_tablet_manager.cc:1428] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260806 00:51:30.191883 4173 raft_consensus.cc:348] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } } peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } }
I20260806 00:51:30.192688 4173 raft_consensus.cc:374] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:51:30.192752 4173 raft_consensus.cc:729] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c1e7f983def48678fd856d6655b41f9, State: Initialized, Role: FOLLOWER
I20260806 00:51:30.192972 4172 ts_tablet_manager.cc:1428] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:51:30.193145 4173 consensus_queue.cc:260] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } } peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } }
I20260806 00:51:30.193773 4148 heartbeater.cc:503] Master 127.15.147.190:40217 was elected leader, sending a full tablet report...
I20260806 00:51:30.194079 4173 ts_tablet_manager.cc:1428] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
I20260806 00:51:30.256399 4179 raft_consensus.cc:482] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:51:30.256618 4179 raft_consensus.cc:504] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } } peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } }
I20260806 00:51:30.257820 4179 leader_election.cc:288] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c1e7f983def48678fd856d6655b41f9 (127.15.147.131:38769), ee5e9fbd481c4102aa9bcde8485eb84f (127.15.147.130:40387)
I20260806 00:51:30.262384 4169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.258207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:36549 (local address 127.15.147.130:40387)
0806 00:51:30.258350 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:30.258355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:30.258565 (+ 210us) server_negotiation.cc:407] Connection header received
0806 00:51:30.258739 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:30.258742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:30.258794 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:30.258881 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:30.259194 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.259656 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.260813 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.261419 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.261533 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.261608 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:30.262115 (+ 507us) server_negotiation.cc:299] Negotiation successful
0806 00:51:30.262212 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:51:30.262826 4162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.258189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:55547 (local address 127.15.147.131:38769)
0806 00:51:30.258343 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:30.258348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:30.258591 (+ 243us) server_negotiation.cc:407] Connection header received
0806 00:51:30.258758 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:30.258762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:30.258812 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:30.258895 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:30.259225 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.259674 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.260833 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.261516 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.261675 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.261741 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:30.262461 (+ 720us) server_negotiation.cc:299] Negotiation successful
0806 00:51:30.262621 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 00:51:30.262894 4185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.258065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:40387 (local address 127.15.147.129:36549)
0806 00:51:30.258468 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:30.258479 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:30.258610 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:30.258894 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:30.258897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:30.258913 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:30.259088 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:30.259093 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.259761 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.259764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:30.260679 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:30.260688 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.261740 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.261747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:30.261934 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.261975 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 00:51:30.262586 (+ 611us) client_negotiation.cc:240] Negotiation successful
0806 00:51:30.262678 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":35,"spinlock_wait_cycles":27008,"thread_start_us":137,"threads_started":1}
I20260806 00:51:30.263501 4184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.258070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:38769 (local address 127.15.147.129:55547)
0806 00:51:30.258490 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:30.258502 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:30.258632 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:30.258908 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:30.258911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:30.258927 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:30.259121 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:30.259126 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.259784 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.259787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:30.260702 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:30.260712 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.261868 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.261871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:30.262197 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.262266 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 00:51:30.263059 (+ 793us) client_negotiation.cc:240] Negotiation successful
0806 00:51:30.263217 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":167,"thread_start_us":121,"threads_started":1}
I20260806 00:51:30.263639 3895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b09bf4f0c1b147c0897d960285a95221" candidate_uuid: "cd7245054dd14610861e2ba0e7821a48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" is_pre_election: true
I20260806 00:51:30.264238 3895 raft_consensus.cc:2454] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd7245054dd14610861e2ba0e7821a48 in term 0.
I20260806 00:51:30.264797 3612 leader_election.cc:302] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd7245054dd14610861e2ba0e7821a48, ee5e9fbd481c4102aa9bcde8485eb84f; no voters:
I20260806 00:51:30.265385 4087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b09bf4f0c1b147c0897d960285a95221" candidate_uuid: "cd7245054dd14610861e2ba0e7821a48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c1e7f983def48678fd856d6655b41f9" is_pre_election: true
I20260806 00:51:30.265530 4179 raft_consensus.cc:2781] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:51:30.265612 4179 raft_consensus.cc:482] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:51:30.265653 4179 raft_consensus.cc:3037] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:30.265681 4087 raft_consensus.cc:2454] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd7245054dd14610861e2ba0e7821a48 in term 0.
I20260806 00:51:30.267331 4179 raft_consensus.cc:504] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } } peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } }
I20260806 00:51:30.267815 4179 leader_election.cc:288] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 1 election: Requested vote from peers 4c1e7f983def48678fd856d6655b41f9 (127.15.147.131:38769), ee5e9fbd481c4102aa9bcde8485eb84f (127.15.147.130:40387)
I20260806 00:51:30.268177 4087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b09bf4f0c1b147c0897d960285a95221" candidate_uuid: "cd7245054dd14610861e2ba0e7821a48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c1e7f983def48678fd856d6655b41f9"
I20260806 00:51:30.268200 3895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b09bf4f0c1b147c0897d960285a95221" candidate_uuid: "cd7245054dd14610861e2ba0e7821a48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f"
I20260806 00:51:30.268293 4087 raft_consensus.cc:3037] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:30.268326 3895 raft_consensus.cc:3037] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:30.269599 4087 raft_consensus.cc:2454] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd7245054dd14610861e2ba0e7821a48 in term 1.
I20260806 00:51:30.269599 3895 raft_consensus.cc:2454] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd7245054dd14610861e2ba0e7821a48 in term 1.
I20260806 00:51:30.269909 3616 leader_election.cc:302] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4c1e7f983def48678fd856d6655b41f9, cd7245054dd14610861e2ba0e7821a48, ee5e9fbd481c4102aa9bcde8485eb84f; no voters:
I20260806 00:51:30.270048 4179 raft_consensus.cc:2781] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:51:30.270293 4179 raft_consensus.cc:686] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [term 1 LEADER]: Becoming Leader. State: Replica: cd7245054dd14610861e2ba0e7821a48, State: Running, Role: LEADER
I20260806 00:51:30.270675 4179 consensus_queue.cc:237] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } } peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } }
I20260806 00:51:30.274967 3500 catalog_manager.cc:5495] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd7245054dd14610861e2ba0e7821a48 (127.15.147.129). New cstate: current_term: 1 leader_uuid: "cd7245054dd14610861e2ba0e7821a48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } health_report { overall_health: UNKNOWN } } }
I20260806 00:51:30.299582 4166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.295639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38228 (local address 127.15.147.129:42081)
0806 00:51:30.295765 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:30.295776 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:30.295855 (+ 79us) server_negotiation.cc:407] Connection header received
0806 00:51:30.296062 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:30.296066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:30.296138 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:30.296248 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:51:30.296597 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.297278 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.298102 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.298311 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.298385 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.298641 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:51:30.299048 (+ 407us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:51:30.299212 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:30.299360 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 00:51:30.299460 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:51:30.305471 3895 raft_consensus.cc:1261] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f [term 1 FOLLOWER]: Refusing update from remote peer cd7245054dd14610861e2ba0e7821a48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:51:30.305471 4087 raft_consensus.cc:1261] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9 [term 1 FOLLOWER]: Refusing update from remote peer cd7245054dd14610861e2ba0e7821a48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:51:30.306278 4186 consensus_queue.cc:1035] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:51:30.306456 4179 consensus_queue.cc:1035] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:51:30.313608 4191 mvcc.cc:205] Tried to move back new op lower bound from 7315363800283680768 to 7315363800151842816. Current Snapshot: MvccSnapshot[applied={T|T < 7315363800283680768}]
I20260806 00:51:30.313786 4193 mvcc.cc:205] Tried to move back new op lower bound from 7315363800283680768 to 7315363800151842816. Current Snapshot: MvccSnapshot[applied={T|T < 7315363800283680768}]
I20260806 00:51:30.314390 4192 mvcc.cc:205] Tried to move back new op lower bound from 7315363800283680768 to 7315363800151842816. Current Snapshot: MvccSnapshot[applied={T|T < 7315363800283680768}]
I20260806 00:51:30.322232 3500 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:51:30.322383 3500 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:51:30.324719 3500 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014: Sending DeleteTablet for 3 replicas of tablet b09bf4f0c1b147c0897d960285a95221
I20260806 00:51:30.325497 4051 tablet_service.cc:1514] Processing DeleteTablet for tablet b09bf4f0c1b147c0897d960285a95221 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:51:30 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:43166
I20260806 00:51:30.325515 3867 tablet_service.cc:1514] Processing DeleteTablet for tablet b09bf4f0c1b147c0897d960285a95221 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:51:30 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:59482
I20260806 00:51:30.325956 4205 tablet_replica.cc:330] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9: stopping tablet replica
I20260806 00:51:30.325956 4207 tablet_replica.cc:330] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f: stopping tablet replica
I20260806 00:51:30.326107 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 3587
I20260806 00:51:30.326247 4207 raft_consensus.cc:2229] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:51:30.326300 4205 raft_consensus.cc:2229] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:51:30.326529 4207 raft_consensus.cc:2258] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:51:30.326555 4205 raft_consensus.cc:2258] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:51:30.329339 4207 ts_tablet_manager.cc:1905] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:51:30.331305 4205 ts_tablet_manager.cc:1905] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:51:30.331866 4207 ts_tablet_manager.cc:1918] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:51:30.331936 4207 log.cc:1196] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/b09bf4f0c1b147c0897d960285a95221
I20260806 00:51:30.332180 4207 ts_tablet_manager.cc:1939] T b09bf4f0c1b147c0897d960285a95221 P ee5e9fbd481c4102aa9bcde8485eb84f: Deleting consensus metadata
I20260806 00:51:30.333700 3481 catalog_manager.cc:4841] TS ee5e9fbd481c4102aa9bcde8485eb84f (127.15.147.130:40387): tablet b09bf4f0c1b147c0897d960285a95221 (table test-table [id=02bb1f01d93041838b47d899a3c579fe]) successfully deleted
I20260806 00:51:30.334683 4205 ts_tablet_manager.cc:1918] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:51:30.334776 4205 log.cc:1196] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/b09bf4f0c1b147c0897d960285a95221
I20260806 00:51:30.335116 4205 ts_tablet_manager.cc:1939] T b09bf4f0c1b147c0897d960285a95221 P 4c1e7f983def48678fd856d6655b41f9: Deleting consensus metadata
I20260806 00:51:30.339103 3485 catalog_manager.cc:4841] TS 4c1e7f983def48678fd856d6655b41f9 (127.15.147.131:38769): tablet b09bf4f0c1b147c0897d960285a95221 (table test-table [id=02bb1f01d93041838b47d899a3c579fe]) successfully deleted
W20260806 00:51:30.345438 3485 connection.cc:535] client connection to 127.15.147.129:42081 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 00:51:30.345624 3485 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 00:51:30.345631 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 3784
I20260806 00:51:30.348935 4164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.348554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:42081 (local address 127.0.0.1:38244)
0806 00:51:30.348764 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:30.348837 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:42081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260806 00:51:30.349086 3485 catalog_manager.cc:4584] TS cd7245054dd14610861e2ba0e7821a48 (127.15.147.129:42081): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b09bf4f0c1b147c0897d960285a95221: Network error: Client connection negotiation failed: client connection to 127.15.147.129:42081: connect: Connection refused (error 111)
I20260806 00:51:30.354775 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 3963
I20260806 00:51:30.370102 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 3459
I20260806 00:51:30.384495 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:40217
--webserver_interface=127.15.147.190
--webserver_port=38655
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:42903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:40217
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:51:30.494976 4212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:30.495204 4212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:30.495239 4212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:30.499159 4212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:51:30.499208 4212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:30.499226 4212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:30.499241 4212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:51:30.499256 4212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:51:30.504372 4212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:42903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:40217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:40217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=38655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:30.505343 4212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:30.506299 4212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:30.520354 4221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:30.520558 4212 server_base.cc:1034] running on GCE node
W20260806 00:51:30.520382 4220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:30.520365 4224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:30.521317 4212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:30.522476 4212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:30.523664 4212 hybrid_clock.cc:648] HybridClock initialized: now 1785977490523653 us; error 29 us; skew 500 ppm
Aug 06 00:51:30 dist-test-slave-45dx krb5kdc[3445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977490, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:30.526443 4212 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:51:30.527309 4212 webserver.cc:460] Webserver started at http://127.15.147.190:38655/ using document root <none> and password file <none>
I20260806 00:51:30.527858 4212 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:30.527930 4212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:30.531838 4212 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:51:30.534099 4234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:30.535475 4212 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:51:30.535619 4212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "5d7cf4b210e843a9b43f587708ec8014"
format_stamp: "Formatted at 2026-08-06 00:51:29 on dist-test-slave-45dx"
server_key: "6b7f4f6aed4968d1a6a5caa1ff81f75c"
server_key_iv: "dcb8346baad17a609f930fea2e42e971"
server_key_version: "encryptionkey@0"
I20260806 00:51:30.536509 4212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:30.550173 4212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:30.553032 4212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:30.553239 4212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:30.560650 4212 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:40217
I20260806 00:51:30.560961 4311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:40217 every 8 connection(s)
I20260806 00:51:30.561846 4212 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:51:30.566282 4312 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:51:30.567956 4312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014: Bootstrap starting.
I20260806 00:51:30.571377 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 4212
I20260806 00:51:30.571833 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:42081
--local_ip_for_outbound_sockets=127.15.147.129
--tserver_master_addrs=127.15.147.190:40217
--webserver_port=37443
--webserver_interface=127.15.147.129
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:42903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:51:30.586431 4312 log.cc:824] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:30.595136 4312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:51:30.595503 4312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014: Bootstrap complete.
I20260806 00:51:30.599624 4312 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7cf4b210e843a9b43f587708ec8014" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40217 } }
I20260806 00:51:30.600047 4312 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d7cf4b210e843a9b43f587708ec8014, State: Initialized, Role: FOLLOWER
I20260806 00:51:30.600586 4312 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7cf4b210e843a9b43f587708ec8014" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40217 } }
I20260806 00:51:30.600731 4312 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:51:30.600824 4312 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:51:30.600945 4312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 1 FOLLOWER]: Advancing to term 2
I20260806 00:51:30.602777 4312 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7cf4b210e843a9b43f587708ec8014" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40217 } }
I20260806 00:51:30.603117 4312 leader_election.cc:302] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d7cf4b210e843a9b43f587708ec8014; no voters:
I20260806 00:51:30.603425 4312 leader_election.cc:288] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 00:51:30.603569 4318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 00:51:30.603820 4318 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 2 LEADER]: Becoming Leader. State: Replica: 5d7cf4b210e843a9b43f587708ec8014, State: Running, Role: LEADER
I20260806 00:51:30.604154 4318 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7cf4b210e843a9b43f587708ec8014" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40217 } }
I20260806 00:51:30.604633 4312 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:51:30.605597 4319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5d7cf4b210e843a9b43f587708ec8014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7cf4b210e843a9b43f587708ec8014" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40217 } } }
I20260806 00:51:30.605742 4319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [sys.catalog]: This master's current role is: LEADER
I20260806 00:51:30.606119 4320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d7cf4b210e843a9b43f587708ec8014. Latest consensus state: current_term: 2 leader_uuid: "5d7cf4b210e843a9b43f587708ec8014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7cf4b210e843a9b43f587708ec8014" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40217 } } }
I20260806 00:51:30.606274 4320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [sys.catalog]: This master's current role is: LEADER
I20260806 00:51:30.606612 4327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:51:30.610491 4327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:51:30.611011 4327 catalog_manager.cc:1259] Loaded cluster ID: fa26c7676ceb45069c6b3eb19d06ce20
I20260806 00:51:30.611088 4327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:51:30.612176 4327 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 00:51:30.614923 4327 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 00:51:30.619390 4327 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:51:30.620030 4327 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014: Loaded TSK: 0
I20260806 00:51:30.621044 4327 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:51:30.699157 4315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:30.699437 4315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:30.699541 4315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:30.703523 4315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:30.703604 4315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:30.703687 4315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:51:30.709241 4315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:42903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:42081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=37443
--tserver_master_addrs=127.15.147.190:40217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:30.710192 4315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:30.711161 4315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:30.721220 4348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:30.721249 4347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:30.721458 4315 server_base.cc:1034] running on GCE node
W20260806 00:51:30.721630 4350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:30.722075 4315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:30.722834 4315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:30.724071 4315 hybrid_clock.cc:648] HybridClock initialized: now 1785977490724047 us; error 32 us; skew 500 ppm
Aug 06 00:51:30 dist-test-slave-45dx krb5kdc[3445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977490, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:30.727417 4315 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:51:30.728279 4315 webserver.cc:460] Webserver started at http://127.15.147.129:37443/ using document root <none> and password file <none>
I20260806 00:51:30.728878 4315 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:30.728982 4315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:30.732897 4315 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:51:30.735069 4357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:30.736390 4315 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:51:30.736521 4315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "cd7245054dd14610861e2ba0e7821a48"
format_stamp: "Formatted at 2026-08-06 00:51:29 on dist-test-slave-45dx"
server_key: "2503862d01c992cb028f7db7f939a878"
server_key_iv: "8cf9a9a34db740548bf63921e1552486"
server_key_version: "encryptionkey@0"
I20260806 00:51:30.737043 4315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:30.753232 4315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:30.756639 4315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:30.756855 4315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:30.757465 4315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:51:30.758683 4367 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 00:51:30.762913 4315 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 00:51:30.762979 4315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260806 00:51:30.763049 4315 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 00:51:30.764781 4315 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 00:51:30.764842 4315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260806 00:51:30.765120 4367 tablet_bootstrap.cc:492] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48: Bootstrap starting.
I20260806 00:51:30.776479 4315 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:42081
I20260806 00:51:30.776539 4525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:42081 every 8 connection(s)
I20260806 00:51:30.777696 4315 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:51:30.779963 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 4315
I20260806 00:51:30.780601 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:40387
--local_ip_for_outbound_sockets=127.15.147.130
--tserver_master_addrs=127.15.147.190:40217
--webserver_port=44453
--webserver_interface=127.15.147.130
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:42903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:51:30.787518 4367 log.cc:824] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:51:30 dist-test-slave-45dx krb5kdc[3445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977490, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:51:30.795128 4367 tablet_bootstrap.cc:492] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:51:30.795593 4367 tablet_bootstrap.cc:492] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48: Bootstrap complete.
I20260806 00:51:30.796907 4531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.779969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:59415)
0806 00:51:30.780869 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:30.780929 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:30.782197 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:30.783566 (+ 1369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:30.783575 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:30.783998 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:30.784516 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:30.784532 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.785829 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.785832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:30.786668 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:30.786678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.787415 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.787420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:30.787645 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.788402 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:30.788428 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:30.790369 (+ 1941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:30.793237 (+ 2868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:30.793242 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:30.793253 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:30.793597 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:30.794053 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:30.794056 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:30.794058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:30.794223 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:30.795042 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:30.795048 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:30.795333 (+ 285us) client_negotiation.cc:769] Sending connection context
0806 00:51:30.795542 (+ 209us) client_negotiation.cc:240] Negotiation successful
0806 00:51:30.795857 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":168,"threads_started":1}
I20260806 00:51:30.797154 4367 ts_tablet_manager.cc:1397] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48: Time spent bootstrapping tablet: real 0.032s user 0.018s sys 0.013s
I20260806 00:51:30.798352 4530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.779973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:59415 (local address 127.15.147.190:40217)
0806 00:51:30.780521 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:30.780544 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:30.781089 (+ 545us) server_negotiation.cc:407] Connection header received
0806 00:51:30.782738 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:30.782755 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:30.783173 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:30.783508 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:30.784695 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.785653 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.786803 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.787262 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.787454 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.790601 (+ 3147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:30.790611 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:30.790630 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:30.790670 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:30.793099 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:30.793746 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:30.793756 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:30.793764 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:30.793936 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:30.794346 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:30.794353 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:30.794357 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:30.794903 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:30.795065 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:30.796576 (+ 1511us) server_negotiation.cc:299] Negotiation successful
0806 00:51:30.796945 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":115,"threads_started":1}
I20260806 00:51:30.799407 4367 raft_consensus.cc:348] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } } peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } }
I20260806 00:51:30.799815 4367 raft_consensus.cc:729] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd7245054dd14610861e2ba0e7821a48, State: Initialized, Role: FOLLOWER
I20260806 00:51:30.800359 4367 consensus_queue.cc:260] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } } peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } }
I20260806 00:51:30.800765 4526 heartbeater.cc:347] Connected to a master server at 127.15.147.190:40217
I20260806 00:51:30.800995 4526 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:30.801357 4367 ts_tablet_manager.cc:1428] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 00:51:30.801513 4526 heartbeater.cc:511] Master 127.15.147.190:40217 requested a full tablet report, sending...
I20260806 00:51:30.804090 4256 ts_manager.cc:194] Registered new tserver with Master: cd7245054dd14610861e2ba0e7821a48 (127.15.147.129:42081)
I20260806 00:51:30.808020 4256 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:59415
I20260806 00:51:30.810482 4526 heartbeater.cc:503] Master 127.15.147.190:40217 was elected leader, sending a full tablet report...
Aug 06 00:51:30 dist-test-slave-45dx krb5kdc[3445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977490, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for kudu/127.15.147.129@KRBTEST.COM
I20260806 00:51:30.818051 4535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.807352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:42081 (local address 127.0.0.1:38246)
0806 00:51:30.808232 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:30.808272 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:30.808464 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:30.808955 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:30.808961 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:30.809388 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:30.809513 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:30.809521 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.810915 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.810919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:30.811543 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:30.811552 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.812255 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.812258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:30.812432 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.812977 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:30.812989 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:30.814301 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:30.816201 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:30.816212 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:30.816217 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:30.816520 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:30.816861 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:30.816864 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:30.816866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:30.816956 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:30.817415 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:30.817420 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:30.817544 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 00:51:30.817672 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 00:51:30.817875 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":85,"threads_started":1}
I20260806 00:51:30.818068 4536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.807400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38246 (local address 127.15.147.129:42081)
0806 00:51:30.807982 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:30.807991 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:30.808395 (+ 404us) server_negotiation.cc:407] Connection header received
0806 00:51:30.808607 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:30.808612 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:30.808803 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:30.808940 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:30.809645 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.810785 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.811667 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.812153 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.812254 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.814436 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:30.814445 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:30.814457 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:30.814484 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:30.816084 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:30.816651 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:30.816656 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:30.816657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:30.816751 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:30.817056 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:30.817059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:30.817061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:30.817284 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:30.817427 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:30.817769 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 00:51:30.817923 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":120,"threads_started":1}
I20260806 00:51:30.818979 4433 tablet_service.cc:1514] Processing DeleteTablet for tablet b09bf4f0c1b147c0897d960285a95221 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:51:30 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:38246
I20260806 00:51:30.819916 4537 tablet_replica.cc:330] stopping tablet replica
I20260806 00:51:30.820199 4537 raft_consensus.cc:2229] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:51:30.820463 4537 raft_consensus.cc:2258] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:51:30.821396 4537 ts_tablet_manager.cc:1905] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:51:30.824898 4537 ts_tablet_manager.cc:1918] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:51:30.825014 4537 log.cc:1196] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/wal/wals/b09bf4f0c1b147c0897d960285a95221
I20260806 00:51:30.825291 4537 ts_tablet_manager.cc:1939] T b09bf4f0c1b147c0897d960285a95221 P cd7245054dd14610861e2ba0e7821a48: Deleting consensus metadata
I20260806 00:51:30.826828 4238 catalog_manager.cc:4841] TS cd7245054dd14610861e2ba0e7821a48 (127.15.147.129:42081): tablet b09bf4f0c1b147c0897d960285a95221 (table test-table [id=02bb1f01d93041838b47d899a3c579fe]) successfully deleted
W20260806 00:51:30.904919 4532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:30.905165 4532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:30.905205 4532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:30.909257 4532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:30.909313 4532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:30.909417 4532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:51:30.914460 4532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:42903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:40387
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=44453
--tserver_master_addrs=127.15.147.190:40217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:30.915429 4532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:30.916458 4532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:30.925937 4543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:30.925961 4547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:30.926038 4532 server_base.cc:1034] running on GCE node
W20260806 00:51:30.925983 4545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:30.926616 4532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:30.927212 4532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:30.928385 4532 hybrid_clock.cc:648] HybridClock initialized: now 1785977490928368 us; error 26 us; skew 500 ppm
Aug 06 00:51:30 dist-test-slave-45dx krb5kdc[3445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977490, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:30.931058 4532 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:51:30.931881 4532 webserver.cc:460] Webserver started at http://127.15.147.130:44453/ using document root <none> and password file <none>
I20260806 00:51:30.932440 4532 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:30.932521 4532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:30.936054 4532 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:51:30.937969 4559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:30.939195 4532 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:51:30.939342 4532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "ee5e9fbd481c4102aa9bcde8485eb84f"
format_stamp: "Formatted at 2026-08-06 00:51:29 on dist-test-slave-45dx"
server_key: "6b236eccd7116b97657b0ad6e47dfc84"
server_key_iv: "2f4e0301cc0220f33259a79f9fad3750"
server_key_version: "encryptionkey@0"
I20260806 00:51:30.939884 4532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:30.961716 4532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:30.965562 4532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:30.965791 4532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:30.966398 4532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:51:30.967267 4532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:51:30.967316 4532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:30.967386 4532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:51:30.967419 4532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:30.976498 4532 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:40387
I20260806 00:51:30.976567 4716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:40387 every 8 connection(s)
I20260806 00:51:30.977582 4532 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:51:30.983196 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 4532
I20260806 00:51:30.983620 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:38769
--local_ip_for_outbound_sockets=127.15.147.131
--tserver_master_addrs=127.15.147.190:40217
--webserver_port=45663
--webserver_interface=127.15.147.131
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:42903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:51:30 dist-test-slave-45dx krb5kdc[3445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977490, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:51:30.992946 4530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.979431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:43871 (local address 127.15.147.190:40217)
0806 00:51:30.979583 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:30.979587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:30.980437 (+ 850us) server_negotiation.cc:407] Connection header received
0806 00:51:30.981874 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:30.981878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:30.981941 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:30.982026 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:30.983386 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.984013 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.985810 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.986231 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.986360 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.988876 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:30.988883 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:30.988889 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:30.988925 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:30.990625 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:30.991118 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:30.991123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:30.991126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:30.991225 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:30.991576 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:30.991581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:30.991584 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:30.991877 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:30.991992 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:30.992507 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 00:51:30.992681 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 00:51:30.993531 4722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:30.979796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:43871)
0806 00:51:30.980248 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:30.980292 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:30.981647 (+ 1355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:30.982194 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:30.982200 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:30.982547 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:30.983160 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:30.983183 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.984149 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.984155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:30.985659 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:30.985675 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:30.986387 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:30.986390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:30.986537 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:30.987185 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:30.987200 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:30.988677 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:30.990743 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:30.990747 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:30.990757 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:30.991002 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:30.991322 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:30.991324 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:30.991325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:30.991451 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:30.991986 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:30.991991 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:30.992202 (+ 211us) client_negotiation.cc:769] Sending connection context
0806 00:51:30.992400 (+ 198us) client_negotiation.cc:240] Negotiation successful
0806 00:51:30.992653 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":122,"threads_started":1}
I20260806 00:51:30.994689 4718 heartbeater.cc:347] Connected to a master server at 127.15.147.190:40217
I20260806 00:51:30.994940 4718 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:30.995419 4718 heartbeater.cc:511] Master 127.15.147.190:40217 requested a full tablet report, sending...
I20260806 00:51:30.996505 4256 ts_manager.cc:194] Registered new tserver with Master: ee5e9fbd481c4102aa9bcde8485eb84f (127.15.147.130:40387)
I20260806 00:51:30.997048 4256 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:43871
W20260806 00:51:31.097347 4723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:31.097630 4723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:31.097705 4723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:31.101625 4723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:31.101712 4723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:31.101826 4723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:51:31.107532 4723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:42903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:38769
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=45663
--tserver_master_addrs=127.15.147.190:40217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:31.108668 4723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:31.109694 4723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:31.119592 4735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:31.119653 4731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:31.119650 4730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:31.119920 4723 server_base.cc:1034] running on GCE node
I20260806 00:51:31.120365 4723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:31.120910 4723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:31.122084 4723 hybrid_clock.cc:648] HybridClock initialized: now 1785977491122069 us; error 24 us; skew 500 ppm
Aug 06 00:51:31 dist-test-slave-45dx krb5kdc[3445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977491, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:31.124934 4723 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:51:31.125753 4723 webserver.cc:460] Webserver started at http://127.15.147.131:45663/ using document root <none> and password file <none>
I20260806 00:51:31.126297 4723 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:31.126405 4723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:31.130098 4723 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 00:51:31.132129 4744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:31.133283 4723 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:51:31.133417 4723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "4c1e7f983def48678fd856d6655b41f9"
format_stamp: "Formatted at 2026-08-06 00:51:30 on dist-test-slave-45dx"
server_key: "f7fe7360381dccaf9aafe5e7eee85699"
server_key_iv: "6665caff47f966b534c70edadebaf320"
server_key_version: "encryptionkey@0"
I20260806 00:51:31.133934 4723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:31.168953 4723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:31.172567 4723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:31.172775 4723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:31.173396 4723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:51:31.174271 4723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:51:31.174345 4723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:31.174418 4723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:51:31.174470 4723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:31.183913 4723 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:38769
I20260806 00:51:31.184000 4905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:38769 every 8 connection(s)
I20260806 00:51:31.184947 4723 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:51:31.192252 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 4723
Aug 06 00:51:31 dist-test-slave-45dx krb5kdc[3445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977491, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:51:31.199404 4530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:31.186744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:55051 (local address 127.15.147.190:40217)
0806 00:51:31.186884 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:31.186888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:31.187722 (+ 834us) server_negotiation.cc:407] Connection header received
0806 00:51:31.189257 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:31.189261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:31.189320 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:31.189434 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:31.190556 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.191073 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.192164 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.192467 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.192578 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:31.195368 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:31.195372 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:31.195374 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:31.195402 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:31.197287 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:31.197812 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:31.197816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:31.197818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:31.197909 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:31.198325 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:31.198327 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:31.198329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:31.198495 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:31.198604 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:31.199100 (+ 496us) server_negotiation.cc:299] Negotiation successful
0806 00:51:31.199233 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:51:31.200376 4911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:31.187116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:55051)
0806 00:51:31.187560 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:31.187620 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:31.189008 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:31.189615 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:31.189621 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:31.189958 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:31.190402 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:31.190415 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.191207 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.191210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:31.192036 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:31.192045 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.192620 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.192624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:31.192789 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:31.193493 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:31.193508 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:31.195206 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:31.197416 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:31.197420 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:31.197430 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:31.197704 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:31.198013 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:31.198015 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:31.198016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:31.198221 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:31.198623 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:31.198630 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:31.198873 (+ 243us) client_negotiation.cc:769] Sending connection context
0806 00:51:31.199055 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 00:51:31.199359 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":118,"threads_started":1}
I20260806 00:51:31.201635 4908 heartbeater.cc:347] Connected to a master server at 127.15.147.190:40217
I20260806 00:51:31.201822 4913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:31.192932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33214 (local address 127.15.147.190:40217)
0806 00:51:31.193207 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:31.193212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:31.193305 (+ 93us) server_negotiation.cc:407] Connection header received
0806 00:51:31.193473 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:31.193476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:31.193521 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:31.193621 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:31.194488 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.195030 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.196194 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.196562 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.196655 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:31.198122 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:31.198126 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:31.198128 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:31.198153 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:31.199839 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:31.200350 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:31.200353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:31.200355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:31.200483 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:31.200851 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:31.200855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:31.200857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:31.201084 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:31.201306 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:31.201514 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 00:51:31.201649 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":93,"threads_started":1}
I20260806 00:51:31.201889 4908 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:31.202412 4908 heartbeater.cc:511] Master 127.15.147.190:40217 requested a full tablet report, sending...
I20260806 00:51:31.203481 4255 ts_manager.cc:194] Registered new tserver with Master: 4c1e7f983def48678fd856d6655b41f9 (127.15.147.131:38769)
I20260806 00:51:31.204033 4255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:55051
I20260806 00:51:31.215258 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:51:31.219123 4913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:31.215829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33218 (local address 127.15.147.190:40217)
0806 00:51:31.215950 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:31.215954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:31.216161 (+ 207us) server_negotiation.cc:407] Connection header received
0806 00:51:31.216279 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:31.216281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:31.216419 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:31.216500 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:51:31.216704 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.217156 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.217965 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.218183 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.218261 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:31.218522 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:51:31.218851 (+ 329us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:51:31.218934 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:31.218996 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 00:51:31.219039 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:51:31.219710 4255 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:33218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:51:31.222801 4255 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:51:31.230856 4433 tablet_service.cc:1467] Processing CreateTablet for tablet 464e11962542489482f8d1e0b4b29e09 (DEFAULT_TABLE table=test-table [id=db76ec1184ad4b08b6427dc053cceef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:51:31.231563 4433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 464e11962542489482f8d1e0b4b29e09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:31.234565 4367 tablet_bootstrap.cc:492] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48: Bootstrap starting.
I20260806 00:51:31.234678 4535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:31.229429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:40387 (local address 127.0.0.1:59486)
0806 00:51:31.229583 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:31.229596 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:31.229664 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:31.230533 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:31.230536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:31.230551 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:31.230647 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:31.230652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.231942 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.231945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:31.233099 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:31.233111 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.234272 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.234276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:31.234370 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:31.234428 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 00:51:31.234480 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 00:51:31.234555 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 00:51:31.235407 4922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:31.229881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:38769 (local address 127.0.0.1:43168)
0806 00:51:31.230299 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:31.230313 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:31.230406 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:31.231290 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:31.231293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:31.231312 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:31.231388 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:31.231394 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.232737 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.232743 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:31.233660 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:31.233671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.235053 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.235056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:31.235135 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:31.235154 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 00:51:31.235215 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 00:51:31.235278 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":101,"threads_started":1}
I20260806 00:51:31.235495 4923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:31.230057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43168 (local address 127.15.147.131:38769)
0806 00:51:31.230562 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:31.230574 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:31.230594 (+ 20us) server_negotiation.cc:407] Connection header received
0806 00:51:31.230676 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:31.230680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:31.230868 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:31.231034 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:31.231759 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.232610 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.233909 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.234913 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.235065 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:31.235189 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:31.235241 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 00:51:31.235319 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":93,"threads_started":1}
I20260806 00:51:31.235571 4367 tablet_bootstrap.cc:654] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:31.235822 4921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:31.229600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59486 (local address 127.15.147.130:40387)
0806 00:51:31.230022 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:31.230043 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:31.230063 (+ 20us) server_negotiation.cc:407] Connection header received
0806 00:51:31.230128 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:31.230132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:31.230342 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:31.230560 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:31.230744 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.231778 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.233244 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.234106 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.235060 (+ 954us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:31.235188 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:31.235238 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 00:51:31.235321 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":178,"threads_started":1}
I20260806 00:51:31.237115 4367 tablet_bootstrap.cc:492] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48: No bootstrap required, opened a new log
I20260806 00:51:31.237229 4367 ts_tablet_manager.cc:1397] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260806 00:51:31.237677 4367 raft_consensus.cc:348] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } } peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } }
I20260806 00:51:31.237798 4367 raft_consensus.cc:374] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:51:31.237845 4367 raft_consensus.cc:729] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd7245054dd14610861e2ba0e7821a48, State: Initialized, Role: FOLLOWER
I20260806 00:51:31.237982 4367 consensus_queue.cc:260] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } } peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } }
I20260806 00:51:31.238349 4367 ts_tablet_manager.cc:1428] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 00:51:31.239190 4814 tablet_service.cc:1467] Processing CreateTablet for tablet 464e11962542489482f8d1e0b4b29e09 (DEFAULT_TABLE table=test-table [id=db76ec1184ad4b08b6427dc053cceef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:51:31.239190 4622 tablet_service.cc:1467] Processing CreateTablet for tablet 464e11962542489482f8d1e0b4b29e09 (DEFAULT_TABLE table=test-table [id=db76ec1184ad4b08b6427dc053cceef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:51:31.240278 4814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 464e11962542489482f8d1e0b4b29e09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:31.240270 4622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 464e11962542489482f8d1e0b4b29e09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:31.248369 4925 tablet_bootstrap.cc:492] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9: Bootstrap starting.
I20260806 00:51:31.248649 4926 tablet_bootstrap.cc:492] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f: Bootstrap starting.
I20260806 00:51:31.250928 4925 tablet_bootstrap.cc:654] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:31.251441 4926 tablet_bootstrap.cc:654] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:31.252327 4926 log.cc:824] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:31.252334 4925 log.cc:824] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:31.254317 4926 tablet_bootstrap.cc:492] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f: No bootstrap required, opened a new log
I20260806 00:51:31.254328 4925 tablet_bootstrap.cc:492] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9: No bootstrap required, opened a new log
I20260806 00:51:31.254524 4926 ts_tablet_manager.cc:1397] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:51:31.254542 4925 ts_tablet_manager.cc:1397] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:51:31.257270 4925 raft_consensus.cc:348] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } } peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } }
I20260806 00:51:31.257268 4926 raft_consensus.cc:348] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } } peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } }
I20260806 00:51:31.257469 4925 raft_consensus.cc:374] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:51:31.257496 4926 raft_consensus.cc:374] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:51:31.257543 4925 raft_consensus.cc:729] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c1e7f983def48678fd856d6655b41f9, State: Initialized, Role: FOLLOWER
I20260806 00:51:31.257565 4926 raft_consensus.cc:729] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee5e9fbd481c4102aa9bcde8485eb84f, State: Initialized, Role: FOLLOWER
I20260806 00:51:31.258028 4925 consensus_queue.cc:260] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } } peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } }
I20260806 00:51:31.258028 4926 consensus_queue.cc:260] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } } peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } }
I20260806 00:51:31.258688 4908 heartbeater.cc:503] Master 127.15.147.190:40217 was elected leader, sending a full tablet report...
I20260806 00:51:31.258688 4718 heartbeater.cc:503] Master 127.15.147.190:40217 was elected leader, sending a full tablet report...
I20260806 00:51:31.259047 4926 ts_tablet_manager.cc:1428] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260806 00:51:31.259047 4925 ts_tablet_manager.cc:1428] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
W20260806 00:51:31.279155 4528 tablet.cc:2369] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:51:31.436185 4909 tablet.cc:2369] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:51:31.478806 4720 tablet.cc:2369] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:51:31.491978 4929 raft_consensus.cc:482] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:51:31.492187 4929 raft_consensus.cc:504] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } } peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } }
I20260806 00:51:31.493423 4929 leader_election.cc:288] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c1e7f983def48678fd856d6655b41f9 (127.15.147.131:38769), cd7245054dd14610861e2ba0e7821a48 (127.15.147.129:42081)
I20260806 00:51:31.497669 4722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:31.493664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:38769 (local address 127.15.147.130:60305)
0806 00:51:31.493838 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:31.493852 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:31.493982 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:31.494333 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:31.494337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:31.494359 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:31.494512 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:31.494521 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.495467 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.495471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:31.496445 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:31.496455 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.497201 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.497204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:31.497309 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:31.497334 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 00:51:31.497415 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 00:51:31.497510 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"mutex_wait_us":1}
I20260806 00:51:31.497679 4923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:31.493859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:60305 (local address 127.15.147.131:38769)
0806 00:51:31.494023 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:31.494029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:31.494059 (+ 30us) server_negotiation.cc:407] Connection header received
0806 00:51:31.494121 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:31.494123 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:31.494211 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:31.494307 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:31.494663 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.495332 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.496576 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.497080 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.497170 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:31.497214 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:31.497440 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 00:51:31.497529 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:51:31.497983 4935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:31.494007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:42081 (local address 127.15.147.130:40819)
0806 00:51:31.494350 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:31.494363 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:31.494470 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:31.494846 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:31.494848 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:31.494866 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:31.494987 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:31.494993 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.495871 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.495874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:31.496732 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:31.496743 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.497528 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.497532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:31.497645 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:31.497668 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:51:31.497742 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 00:51:31.497825 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":85,"threads_started":1}
I20260806 00:51:31.498245 4936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:31.494125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:40819 (local address 127.15.147.129:42081)
0806 00:51:31.494372 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:31.494376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:31.494457 (+ 81us) server_negotiation.cc:407] Connection header received
0806 00:51:31.494576 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:31.494581 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:31.494687 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:31.494784 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:31.495102 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.495733 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.496869 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.497412 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.497519 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:31.497681 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:31.497903 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 00:51:31.497996 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":77,"threads_started":1}
I20260806 00:51:31.498440 4839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "464e11962542489482f8d1e0b4b29e09" candidate_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c1e7f983def48678fd856d6655b41f9" is_pre_election: true
I20260806 00:51:31.498752 4458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "464e11962542489482f8d1e0b4b29e09" candidate_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd7245054dd14610861e2ba0e7821a48" is_pre_election: true
I20260806 00:51:31.498803 4839 raft_consensus.cc:2454] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5e9fbd481c4102aa9bcde8485eb84f in term 0.
I20260806 00:51:31.499037 4458 raft_consensus.cc:2454] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee5e9fbd481c4102aa9bcde8485eb84f in term 0.
I20260806 00:51:31.499329 4562 leader_election.cc:302] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c1e7f983def48678fd856d6655b41f9, ee5e9fbd481c4102aa9bcde8485eb84f; no voters:
I20260806 00:51:31.499608 4929 raft_consensus.cc:2781] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:51:31.499684 4929 raft_consensus.cc:482] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:51:31.499714 4929 raft_consensus.cc:3037] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:31.500840 4929 raft_consensus.cc:504] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } } peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } }
I20260806 00:51:31.501181 4929 leader_election.cc:288] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [CANDIDATE]: Term 1 election: Requested vote from peers 4c1e7f983def48678fd856d6655b41f9 (127.15.147.131:38769), cd7245054dd14610861e2ba0e7821a48 (127.15.147.129:42081)
I20260806 00:51:31.501600 4839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "464e11962542489482f8d1e0b4b29e09" candidate_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c1e7f983def48678fd856d6655b41f9"
I20260806 00:51:31.501600 4458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "464e11962542489482f8d1e0b4b29e09" candidate_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd7245054dd14610861e2ba0e7821a48"
I20260806 00:51:31.501726 4458 raft_consensus.cc:3037] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:31.501726 4839 raft_consensus.cc:3037] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:31.503028 4458 raft_consensus.cc:2454] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5e9fbd481c4102aa9bcde8485eb84f in term 1.
I20260806 00:51:31.503028 4839 raft_consensus.cc:2454] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee5e9fbd481c4102aa9bcde8485eb84f in term 1.
I20260806 00:51:31.503312 4562 leader_election.cc:302] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c1e7f983def48678fd856d6655b41f9, ee5e9fbd481c4102aa9bcde8485eb84f; no voters:
I20260806 00:51:31.503500 4929 raft_consensus.cc:2781] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:51:31.503729 4929 raft_consensus.cc:686] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [term 1 LEADER]: Becoming Leader. State: Replica: ee5e9fbd481c4102aa9bcde8485eb84f, State: Running, Role: LEADER
I20260806 00:51:31.504032 4929 consensus_queue.cc:237] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } } peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } }
I20260806 00:51:31.507361 4256 catalog_manager.cc:5495] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f reported cstate change: term changed from 0 to 1, leader changed from <none> to ee5e9fbd481c4102aa9bcde8485eb84f (127.15.147.130). New cstate: current_term: 1 leader_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } health_report { overall_health: UNKNOWN } } }
I20260806 00:51:31.529428 4921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:31.525868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59490 (local address 127.15.147.130:40387)
0806 00:51:31.526021 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:31.526025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:31.526039 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:51:31.526136 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:31.526139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:31.526194 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:31.526348 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:51:31.526554 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.527087 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.527897 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:31.528109 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:31.528194 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:31.528512 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:51:31.528937 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:51:31.529034 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:31.529151 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 00:51:31.529236 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 00:51:31.534992 4839 raft_consensus.cc:1261] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9 [term 1 FOLLOWER]: Refusing update from remote peer ee5e9fbd481c4102aa9bcde8485eb84f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:51:31.534981 4458 raft_consensus.cc:1261] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Refusing update from remote peer ee5e9fbd481c4102aa9bcde8485eb84f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:51:31.535686 4929 consensus_queue.cc:1035] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:51:31.535905 4937 consensus_queue.cc:1035] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:51:31.542912 4940 mvcc.cc:205] Tried to move back new op lower bound from 7315363805320310784 to 7315363805203529728. Current Snapshot: MvccSnapshot[applied={T|T < 7315363805320310784}]
I20260806 00:51:31.552744 4256 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:33218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:51:31.552945 4256 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:33218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:51:31.555414 4256 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014: Sending DeleteTablet for 3 replicas of tablet 464e11962542489482f8d1e0b4b29e09
I20260806 00:51:31.556241 4814 tablet_service.cc:1514] Processing DeleteTablet for tablet 464e11962542489482f8d1e0b4b29e09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:51:31 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:43168
I20260806 00:51:31.556284 4433 tablet_service.cc:1514] Processing DeleteTablet for tablet 464e11962542489482f8d1e0b4b29e09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:51:31 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:38246
I20260806 00:51:31.556398 4622 tablet_service.cc:1514] Processing DeleteTablet for tablet 464e11962542489482f8d1e0b4b29e09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:51:31 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:59486
I20260806 00:51:31.556800 4955 tablet_replica.cc:330] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9: stopping tablet replica
I20260806 00:51:31.556951 4953 tablet_replica.cc:330] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48: stopping tablet replica
I20260806 00:51:31.557067 4953 raft_consensus.cc:2229] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:51:31.557099 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 4315
I20260806 00:51:31.557112 4955 raft_consensus.cc:2229] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:51:31.557191 4953 raft_consensus.cc:2258] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:51:31.585937 4955 raft_consensus.cc:2258] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:51:31.587695 4955 ts_tablet_manager.cc:1905] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:51:31.588634 4957 tablet_replica.cc:330] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f: stopping tablet replica
I20260806 00:51:31.588980 4957 raft_consensus.cc:2229] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:51:31.589387 4957 raft_consensus.cc:2258] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:51:31.590927 4955 ts_tablet_manager.cc:1918] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:51:31.591048 4955 log.cc:1196] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/464e11962542489482f8d1e0b4b29e09
I20260806 00:51:31.591348 4957 ts_tablet_manager.cc:1905] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:51:31.591384 4955 ts_tablet_manager.cc:1939] T 464e11962542489482f8d1e0b4b29e09 P 4c1e7f983def48678fd856d6655b41f9: Deleting consensus metadata
I20260806 00:51:31.592768 4238 catalog_manager.cc:4841] TS 4c1e7f983def48678fd856d6655b41f9 (127.15.147.131:38769): tablet 464e11962542489482f8d1e0b4b29e09 (table test-table [id=db76ec1184ad4b08b6427dc053cceef2]) successfully deleted
I20260806 00:51:31.594662 4957 ts_tablet_manager.cc:1918] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:51:31.594794 4957 log.cc:1196] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/464e11962542489482f8d1e0b4b29e09
I20260806 00:51:31.595152 4957 ts_tablet_manager.cc:1939] T 464e11962542489482f8d1e0b4b29e09 P ee5e9fbd481c4102aa9bcde8485eb84f: Deleting consensus metadata
I20260806 00:51:31.596513 4235 catalog_manager.cc:4841] TS ee5e9fbd481c4102aa9bcde8485eb84f (127.15.147.130:40387): tablet 464e11962542489482f8d1e0b4b29e09 (table test-table [id=db76ec1184ad4b08b6427dc053cceef2]) successfully deleted
W20260806 00:51:31.601248 4238 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.147.129:42081 (error 108)
I20260806 00:51:31.601950 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 4532
I20260806 00:51:31.613276 4922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:31.612945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:42081 (local address 127.0.0.1:38250)
0806 00:51:31.613097 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:31.613174 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:42081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 00:51:31.613487 4238 catalog_manager.cc:4584] TS cd7245054dd14610861e2ba0e7821a48 (127.15.147.129:42081): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 464e11962542489482f8d1e0b4b29e09: Network error: Client connection negotiation failed: client connection to 127.15.147.129:42081: connect: Connection refused (error 111)
I20260806 00:51:31.618690 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 4723
I20260806 00:51:31.633772 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 4212
I20260806 00:51:31.650233 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:40217
--webserver_interface=127.15.147.190
--webserver_port=38655
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:42903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:40217
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:51:31.763572 4959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:31.763923 4959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:31.763993 4959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:31.767848 4959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:51:31.767946 4959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:31.767983 4959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:31.768020 4959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:51:31.768055 4959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:51:31.773289 4959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:42903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:40217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:40217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=38655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:31.774403 4959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:31.775475 4959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:31.783895 4966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:31.783933 4965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:31.784031 4959 server_base.cc:1034] running on GCE node
W20260806 00:51:31.783915 4971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:31.784776 4959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:31.785935 4959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:31.787092 4959 hybrid_clock.cc:648] HybridClock initialized: now 1785977491787063 us; error 43 us; skew 500 ppm
Aug 06 00:51:31 dist-test-slave-45dx krb5kdc[3445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977491, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:31.802326 4959 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:51:31.803417 4959 webserver.cc:460] Webserver started at http://127.15.147.190:38655/ using document root <none> and password file <none>
I20260806 00:51:31.804003 4959 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:31.804100 4959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:31.808039 4959 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:51:31.810173 4980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:31.811358 4959 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:51:31.811506 4959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "5d7cf4b210e843a9b43f587708ec8014"
format_stamp: "Formatted at 2026-08-06 00:51:29 on dist-test-slave-45dx"
server_key: "6b7f4f6aed4968d1a6a5caa1ff81f75c"
server_key_iv: "dcb8346baad17a609f930fea2e42e971"
server_key_version: "encryptionkey@0"
I20260806 00:51:31.812040 4959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:31.833901 4959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:31.837214 4959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:31.837433 4959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:31.845127 4959 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:40217
I20260806 00:51:31.845433 5056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:40217 every 8 connection(s)
I20260806 00:51:31.846293 4959 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:51:31.847383 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 4959
I20260806 00:51:31.847831 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:42081
--local_ip_for_outbound_sockets=127.15.147.129
--tserver_master_addrs=127.15.147.190:40217
--webserver_port=37443
--webserver_interface=127.15.147.129
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:42903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:51:31.851128 5057 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:51:31.852788 5057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014: Bootstrap starting.
I20260806 00:51:31.872006 5057 log.cc:824] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:31.886499 5057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:51:31.886947 5057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014: Bootstrap complete.
I20260806 00:51:31.891213 5057 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7cf4b210e843a9b43f587708ec8014" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40217 } }
I20260806 00:51:31.891647 5057 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d7cf4b210e843a9b43f587708ec8014, State: Initialized, Role: FOLLOWER
I20260806 00:51:31.892145 5057 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7cf4b210e843a9b43f587708ec8014" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40217 } }
I20260806 00:51:31.892297 5057 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:51:31.892426 5057 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:51:31.892537 5057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 2 FOLLOWER]: Advancing to term 3
I20260806 00:51:31.894403 5057 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7cf4b210e843a9b43f587708ec8014" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40217 } }
I20260806 00:51:31.894734 5057 leader_election.cc:302] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d7cf4b210e843a9b43f587708ec8014; no voters:
I20260806 00:51:31.895118 5057 leader_election.cc:288] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 00:51:31.895215 5064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 00:51:31.895427 5064 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [term 3 LEADER]: Becoming Leader. State: Replica: 5d7cf4b210e843a9b43f587708ec8014, State: Running, Role: LEADER
I20260806 00:51:31.895707 5064 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7cf4b210e843a9b43f587708ec8014" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40217 } }
I20260806 00:51:31.896360 5057 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:51:31.897305 5066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5d7cf4b210e843a9b43f587708ec8014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7cf4b210e843a9b43f587708ec8014" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40217 } } }
I20260806 00:51:31.897459 5066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [sys.catalog]: This master's current role is: LEADER
I20260806 00:51:31.898006 5067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d7cf4b210e843a9b43f587708ec8014. Latest consensus state: current_term: 3 leader_uuid: "5d7cf4b210e843a9b43f587708ec8014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d7cf4b210e843a9b43f587708ec8014" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 40217 } } }
I20260806 00:51:31.898166 5067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d7cf4b210e843a9b43f587708ec8014 [sys.catalog]: This master's current role is: LEADER
I20260806 00:51:31.898490 5075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:51:31.903579 5075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:51:31.904191 5075 catalog_manager.cc:1259] Loaded cluster ID: fa26c7676ceb45069c6b3eb19d06ce20
I20260806 00:51:31.904251 5075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 00:51:31.905413 5075 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc4c4eca9ad google::LogMessage::Fail() at ??:0
@ 0x7fc4c4ecd88a google::LogMessage::SendToLog() at ??:0
@ 0x7fc4c4eca4de google::LogMessage::Flush() at ??:0
@ 0x7fc4c4ecba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc4cf809fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fc4cf80ad2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc4cf807579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fc4cf84b640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fc4c711c3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc4c711cc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fc4c711e3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fc4c710d512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc4c87516db start_thread at ??:0
@ 0x7fc4c39c861f clone at ??:0
W20260806 00:51:32.016485 5059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:32.016709 5059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:32.016788 5059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:32.020723 5059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:32.020771 5059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:32.020862 5059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:51:32.025923 5059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:42903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:42081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=37443
--tserver_master_addrs=127.15.147.190:40217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:32.026929 5059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:32.027930 5059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:32.037611 5111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:32.037827 5123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:32.037598 5116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:32.038039 5059 server_base.cc:1034] running on GCE node
I20260806 00:51:32.038658 5059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:32.039464 5059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:32.040695 5059 hybrid_clock.cc:648] HybridClock initialized: now 1785977492040655 us; error 46 us; skew 500 ppm
Aug 06 00:51:32 dist-test-slave-45dx krb5kdc[3445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977492, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:32.044128 5059 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:51:32.045042 5059 webserver.cc:460] Webserver started at http://127.15.147.129:37443/ using document root <none> and password file <none>
I20260806 00:51:32.045580 5059 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:32.045643 5059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:32.049489 5059 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:51:32.051648 5131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:32.052951 5059 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:51:32.053045 5059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "cd7245054dd14610861e2ba0e7821a48"
format_stamp: "Formatted at 2026-08-06 00:51:29 on dist-test-slave-45dx"
server_key: "2503862d01c992cb028f7db7f939a878"
server_key_iv: "8cf9a9a34db740548bf63921e1552486"
server_key_version: "encryptionkey@0"
I20260806 00:51:32.053598 5059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:32.080015 5059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:32.083348 5059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:32.083556 5059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:32.084153 5059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:51:32.085302 5144 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 00:51:32.089679 5059 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 00:51:32.089740 5059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260806 00:51:32.089789 5059 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 00:51:32.091445 5059 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 00:51:32.091503 5059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 00:51:32.091814 5144 tablet_bootstrap.cc:492] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48: Bootstrap starting.
I20260806 00:51:32.102578 5059 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:42081
I20260806 00:51:32.102898 5306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:42081 every 8 connection(s)
I20260806 00:51:32.103933 5059 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:51:32.106899 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 5059
I20260806 00:51:32.107385 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:40387
--local_ip_for_outbound_sockets=127.15.147.130
--tserver_master_addrs=127.15.147.190:40217
--webserver_port=44453
--webserver_interface=127.15.147.130
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:42903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:51:32.114892 5144 log.cc:824] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:32.125079 5144 tablet_bootstrap.cc:492] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:51:32.125541 5144 tablet_bootstrap.cc:492] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48: Bootstrap complete.
I20260806 00:51:32.127211 5144 ts_tablet_manager.cc:1397] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48: Time spent bootstrapping tablet: real 0.036s user 0.020s sys 0.012s
I20260806 00:51:32.130451 5144 raft_consensus.cc:348] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } } peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } }
I20260806 00:51:32.131099 5144 raft_consensus.cc:729] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd7245054dd14610861e2ba0e7821a48, State: Initialized, Role: FOLLOWER
I20260806 00:51:32.131628 5144 consensus_queue.cc:260] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } } peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } }
I20260806 00:51:32.132678 5144 ts_tablet_manager.cc:1428] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
W20260806 00:51:32.146929 5309 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:51:32.106219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:35295)
0806 00:51:32.108758 (+ 2539us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:32.108816 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:32.110057 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:32.145850 (+ 35793us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":2286,"thread_start_us":197,"threads_started":1}
W20260806 00:51:32.147519 5307 heartbeater.cc:644] Failed to heartbeat to 127.15.147.190:40217 (0 consecutive failures): Network error: Failed to ping master at 127.15.147.190:40217: Client connection negotiation failed: client connection to 127.15.147.190:40217: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 00:51:32.148245 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:32.147989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:33001)
0806 00:51:32.148119 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:32.148196 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 00:51:32.148911 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:32.148700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:56747)
0806 00:51:32.148808 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:32.148855 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 00:51:32.149077 5307 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:51:32.230849 5310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:32.231119 5310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:32.231194 5310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:32.235018 5310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:32.235096 5310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:32.235194 5310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:51:32.240268 5310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:42903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:40387
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=44453
--tserver_master_addrs=127.15.147.190:40217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:32.241290 5310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:32.242326 5310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:32.251737 5317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:32.251829 5310 server_base.cc:1034] running on GCE node
W20260806 00:51:32.251730 5321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:32.251730 5319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:32.252389 5310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:32.252965 5310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:32.254153 5310 hybrid_clock.cc:648] HybridClock initialized: now 1785977492254123 us; error 40 us; skew 500 ppm
Aug 06 00:51:32 dist-test-slave-45dx krb5kdc[3445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977492, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:32.256969 5310 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:51:32.257893 5310 webserver.cc:460] Webserver started at http://127.15.147.130:44453/ using document root <none> and password file <none>
I20260806 00:51:32.258431 5310 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:32.258522 5310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:32.262072 5310 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:51:32.266211 5331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:32.267817 5310 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:51:32.267940 5310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "ee5e9fbd481c4102aa9bcde8485eb84f"
format_stamp: "Formatted at 2026-08-06 00:51:29 on dist-test-slave-45dx"
server_key: "6b236eccd7116b97657b0ad6e47dfc84"
server_key_iv: "2f4e0301cc0220f33259a79f9fad3750"
server_key_version: "encryptionkey@0"
I20260806 00:51:32.268532 5310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:32.286492 5310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:32.289898 5310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:32.290057 5310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:32.290599 5310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:51:32.291617 5310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:51:32.291678 5310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:32.291723 5310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:51:32.291745 5310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:32.301287 5310 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:40387
I20260806 00:51:32.301332 5489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:40387 every 8 connection(s)
I20260806 00:51:32.302297 5310 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:51:32.306044 5492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:32.304287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:47219)
0806 00:51:32.304744 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:32.304856 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":107,"threads_started":1}
W20260806 00:51:32.306579 5490 heartbeater.cc:644] Failed to heartbeat to 127.15.147.190:40217 (0 consecutive failures): Network error: Failed to ping master at 127.15.147.190:40217: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
I20260806 00:51:32.307209 5492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:32.307000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:53673)
0806 00:51:32.307110 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:32.307156 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 00:51:32.307793 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 5310
I20260806 00:51:32.307914 5492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:32.307669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:44297)
0806 00:51:32.307786 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:32.307844 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 00:51:32.308063 5490 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 00:51:32.308341 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:38769
--local_ip_for_outbound_sockets=127.15.147.131
--tserver_master_addrs=127.15.147.190:40217
--webserver_port=45663
--webserver_interface=127.15.147.131
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:42903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:51:32.424551 5493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:32.424798 5493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:32.424865 5493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:32.428835 5493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:32.428891 5493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:32.428959 5493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:51:32.434105 5493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:42903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:38769
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=45663
--tserver_master_addrs=127.15.147.190:40217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:32.435079 5493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:32.436045 5493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:32.445943 5504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:32.445986 5502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:32.446005 5493 server_base.cc:1034] running on GCE node
W20260806 00:51:32.445950 5500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:32.446609 5493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:32.447292 5493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:32.448452 5493 hybrid_clock.cc:648] HybridClock initialized: now 1785977492448432 us; error 29 us; skew 500 ppm
Aug 06 00:51:32 dist-test-slave-45dx krb5kdc[3445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977492, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:32.451174 5493 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:51:32.452041 5493 webserver.cc:460] Webserver started at http://127.15.147.131:45663/ using document root <none> and password file <none>
I20260806 00:51:32.452625 5493 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:32.452692 5493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:32.456452 5493 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:51:32.458643 5515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:32.459892 5493 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:51:32.460031 5493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "4c1e7f983def48678fd856d6655b41f9"
format_stamp: "Formatted at 2026-08-06 00:51:30 on dist-test-slave-45dx"
server_key: "f7fe7360381dccaf9aafe5e7eee85699"
server_key_iv: "6665caff47f966b534c70edadebaf320"
server_key_version: "encryptionkey@0"
I20260806 00:51:32.460636 5493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:32.479027 5493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:32.482137 5493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:32.482349 5493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:32.482940 5493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:51:32.483726 5493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:51:32.483803 5493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:32.484004 5493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:51:32.484094 5493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:32.493582 5493 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:38769
I20260806 00:51:32.493620 5690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:38769 every 8 connection(s)
I20260806 00:51:32.494539 5493 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:51:32.496101 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 5493
I20260806 00:51:32.498374 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:32.496612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:58259)
0806 00:51:32.497051 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:32.497172 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":126,"threads_started":1}
W20260806 00:51:32.498791 5691 heartbeater.cc:644] Failed to heartbeat to 127.15.147.190:40217 (0 consecutive failures): Network error: Failed to ping master at 127.15.147.190:40217: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
I20260806 00:51:32.499377 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:32.499165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:57303)
0806 00:51:32.499268 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:32.499319 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 00:51:32.499943 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:32.499760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:39675)
0806 00:51:32.499850 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:32.499892 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 00:51:32.500116 5691 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 00:51:33.150045 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:33.149627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:38069)
0806 00:51:33.149881 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:33.149930 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":77,"threads_started":1}
I20260806 00:51:33.309150 5697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:33.308697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:47693)
0806 00:51:33.308985 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:33.309042 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":73,"threads_started":1}
I20260806 00:51:33.501140 5698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:33.500753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:42959)
0806 00:51:33.500988 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:33.501042 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":68,"threads_started":1}
I20260806 00:51:33.606616 5699 raft_consensus.cc:482] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:51:33.606895 5699 raft_consensus.cc:504] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } } peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } }
I20260806 00:51:33.608078 5699 leader_election.cc:288] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ee5e9fbd481c4102aa9bcde8485eb84f (127.15.147.130:40387), 4c1e7f983def48678fd856d6655b41f9 (127.15.147.131:38769)
Aug 06 00:51:33 dist-test-slave-45dx krb5kdc[3445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977492, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.131@KRBTEST.COM
Aug 06 00:51:33 dist-test-slave-45dx krb5kdc[3445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977492, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.130@KRBTEST.COM
I20260806 00:51:33.623127 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:33.608366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:38769 (local address 127.15.147.129:46113)
0806 00:51:33.608853 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:33.608865 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:33.608952 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:33.610858 (+ 1906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:33.610864 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:33.611508 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:33.611972 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:33.611994 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:33.613420 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:33.613424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:33.614407 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:33.614417 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:33.615547 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:33.615552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:33.615659 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:33.616054 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:33.616065 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:33.617954 (+ 1889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:33.620339 (+ 2385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:33.620346 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:33.620380 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:33.620746 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:33.621178 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:33.621182 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:33.621184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:33.621372 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:33.622089 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:33.622096 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:33.622371 (+ 275us) client_negotiation.cc:769] Sending connection context
0806 00:51:33.622559 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 00:51:33.622902 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":120,"thread_start_us":99,"threads_started":1}
I20260806 00:51:33.623144 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:33.608530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:46113 (local address 127.15.147.131:38769)
0806 00:51:33.608781 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:33.608810 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:33.608968 (+ 158us) server_negotiation.cc:407] Connection header received
0806 00:51:33.609920 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:33.609928 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:33.610385 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:33.610729 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:33.612172 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:33.613261 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:33.614545 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:33.615184 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:33.615373 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:33.618244 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:33.618253 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:33.618270 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:33.618311 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:33.620189 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:33.620879 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:33.620886 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:33.620892 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:33.621081 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:33.621490 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:33.621493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:33.621495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:33.621955 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:33.622117 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:33.622701 (+ 584us) server_negotiation.cc:299] Negotiation successful
0806 00:51:33.622968 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":62,"threads_started":1}
I20260806 00:51:33.623174 5702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:33.608515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:50583 (local address 127.15.147.130:40387)
0806 00:51:33.608984 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:33.609011 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:33.609045 (+ 34us) server_negotiation.cc:407] Connection header received
0806 00:51:33.609919 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:33.609928 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:33.610385 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:33.611302 (+ 917us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:33.612432 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:33.613457 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:33.614544 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:33.615015 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:33.615216 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:33.618538 (+ 3322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:33.618543 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:33.618554 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:33.618581 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:33.620525 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:33.621064 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:33.621069 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:33.621072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:33.621235 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:33.621606 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:33.621610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:33.621611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:33.622028 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:33.622174 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:33.622683 (+ 509us) server_negotiation.cc:299] Negotiation successful
0806 00:51:33.622975 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":112,"threads_started":1}
I20260806 00:51:33.623162 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:33.608271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:40387 (local address 127.15.147.129:50583)
0806 00:51:33.608412 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:33.608430 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:33.608538 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:33.610832 (+ 2294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:33.610838 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:33.611490 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:33.611973 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:33.611987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:33.613624 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:33.613626 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:33.614404 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:33.614414 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:33.615182 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:33.615185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:33.615364 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:33.615989 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:33.616014 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:33.618343 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:33.620664 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:33.620666 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:33.620668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:33.620939 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:33.621341 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:33.621343 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:33.621345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:33.621476 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:33.622139 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:33.622142 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:33.622372 (+ 230us) client_negotiation.cc:769] Sending connection context
0806 00:51:33.622556 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 00:51:33.622992 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":203,"spinlock_wait_cycles":43520}
W20260806 00:51:33.625808 5136 leader_election.cc:341] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4c1e7f983def48678fd856d6655b41f9 (127.15.147.131:38769): Not found: Tablet not found: 464e11962542489482f8d1e0b4b29e09
W20260806 00:51:33.626003 5133 leader_election.cc:341] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee5e9fbd481c4102aa9bcde8485eb84f (127.15.147.130:40387): Not found: Tablet not found: 464e11962542489482f8d1e0b4b29e09
I20260806 00:51:33.626096 5133 leader_election.cc:302] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd7245054dd14610861e2ba0e7821a48; no voters: 4c1e7f983def48678fd856d6655b41f9, ee5e9fbd481c4102aa9bcde8485eb84f
I20260806 00:51:33.626410 5699 raft_consensus.cc:2726] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:51:34.151202 5703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:34.150803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:59753)
0806 00:51:34.151052 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:34.151104 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
I20260806 00:51:34.310410 5705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:34.310001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:49901)
0806 00:51:34.310265 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:34.310323 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":74,"threads_started":1}
I20260806 00:51:34.502230 5707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:34.501870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:47159)
0806 00:51:34.502098 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:34.502150 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":69,"threads_started":1}
I20260806 00:51:35.152590 5709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:35.152085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:51559)
0806 00:51:35.152421 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:35.152474 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":84,"threads_started":1}
I20260806 00:51:35.311568 5713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:35.311157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:48243)
0806 00:51:35.311408 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:35.311464 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":77,"threads_started":1}
I20260806 00:51:35.478888 5714 raft_consensus.cc:482] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:51:35.479024 5714 raft_consensus.cc:504] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } } peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } }
I20260806 00:51:35.479521 5714 leader_election.cc:288] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ee5e9fbd481c4102aa9bcde8485eb84f (127.15.147.130:40387), 4c1e7f983def48678fd856d6655b41f9 (127.15.147.131:38769)
W20260806 00:51:35.480458 5133 leader_election.cc:341] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee5e9fbd481c4102aa9bcde8485eb84f (127.15.147.130:40387): Not found: Tablet not found: 464e11962542489482f8d1e0b4b29e09
W20260806 00:51:35.480572 5136 leader_election.cc:341] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4c1e7f983def48678fd856d6655b41f9 (127.15.147.131:38769): Not found: Tablet not found: 464e11962542489482f8d1e0b4b29e09
I20260806 00:51:35.480638 5136 leader_election.cc:302] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd7245054dd14610861e2ba0e7821a48; no voters: 4c1e7f983def48678fd856d6655b41f9, ee5e9fbd481c4102aa9bcde8485eb84f
I20260806 00:51:35.480774 5714 raft_consensus.cc:2726] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:51:35.503811 5716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:35.503134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:37229)
0806 00:51:35.503548 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:35.503641 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":134,"threads_started":1}
I20260806 00:51:36.153919 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:36.153368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:41507)
0806 00:51:36.153756 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:36.153818 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":100,"threads_started":1}
I20260806 00:51:36.312742 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:36.312288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:60977)
0806 00:51:36.312599 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:36.312654 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":87,"threads_started":1}
I20260806 00:51:36.505046 5721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:36.504622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:35157)
0806 00:51:36.504893 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:36.504952 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":75,"threads_started":1}
I20260806 00:51:37.155478 5722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:37.154978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:38057)
0806 00:51:37.155325 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:37.155378 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":152,"threads_started":1}
I20260806 00:51:37.313870 5723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:37.313480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:56083)
0806 00:51:37.313723 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:37.313777 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":82,"threads_started":1}
I20260806 00:51:37.506245 5726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:37.505862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:33739)
0806 00:51:37.506105 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:37.506159 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":72,"threads_started":1}
I20260806 00:51:37.878209 5727 raft_consensus.cc:482] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:51:37.878345 5727 raft_consensus.cc:504] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } } peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } }
I20260806 00:51:37.878949 5727 leader_election.cc:288] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ee5e9fbd481c4102aa9bcde8485eb84f (127.15.147.130:40387), 4c1e7f983def48678fd856d6655b41f9 (127.15.147.131:38769)
W20260806 00:51:37.879724 5133 leader_election.cc:341] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee5e9fbd481c4102aa9bcde8485eb84f (127.15.147.130:40387): Not found: Tablet not found: 464e11962542489482f8d1e0b4b29e09
W20260806 00:51:37.879894 5136 leader_election.cc:341] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4c1e7f983def48678fd856d6655b41f9 (127.15.147.131:38769): Not found: Tablet not found: 464e11962542489482f8d1e0b4b29e09
I20260806 00:51:37.879983 5136 leader_election.cc:302] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd7245054dd14610861e2ba0e7821a48; no voters: 4c1e7f983def48678fd856d6655b41f9, ee5e9fbd481c4102aa9bcde8485eb84f
I20260806 00:51:37.880141 5727 raft_consensus.cc:2726] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:51:38.156762 5728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:38.156277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:39397)
0806 00:51:38.156603 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:38.156661 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":76,"threads_started":1}
I20260806 00:51:38.315099 5730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:38.314677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:39187)
0806 00:51:38.314944 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:38.315009 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":97,"threads_started":1}
I20260806 00:51:38.507620 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:38.507190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:38825)
0806 00:51:38.507454 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:38.507528 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":74,"threads_started":1}
I20260806 00:51:39.158010 5732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:39.157590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:55463)
0806 00:51:39.157846 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:39.157911 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":74,"threads_started":1}
I20260806 00:51:39.316382 5733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:39.315883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:41371)
0806 00:51:39.316187 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:39.316240 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":90,"threads_started":1}
I20260806 00:51:39.508876 5739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:39.508427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:57801)
0806 00:51:39.508728 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:39.508783 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":69,"threads_started":1}
I20260806 00:51:40.159173 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:40.158769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:44485)
0806 00:51:40.159014 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:40.159071 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":74,"threads_started":1}
I20260806 00:51:40.317503 5742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:40.317102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:42331)
0806 00:51:40.317345 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:40.317402 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":75,"threads_started":1}
I20260806 00:51:40.510027 5744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:40.509594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:41749)
0806 00:51:40.509865 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:40.509922 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":89,"threads_started":1}
I20260806 00:51:41.008664 5745 raft_consensus.cc:482] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:51:41.008826 5745 raft_consensus.cc:504] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } } peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } }
I20260806 00:51:41.009212 5745 leader_election.cc:288] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ee5e9fbd481c4102aa9bcde8485eb84f (127.15.147.130:40387), 4c1e7f983def48678fd856d6655b41f9 (127.15.147.131:38769)
W20260806 00:51:41.010067 5133 leader_election.cc:341] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee5e9fbd481c4102aa9bcde8485eb84f (127.15.147.130:40387): Not found: Tablet not found: 464e11962542489482f8d1e0b4b29e09
W20260806 00:51:41.010179 5136 leader_election.cc:341] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4c1e7f983def48678fd856d6655b41f9 (127.15.147.131:38769): Not found: Tablet not found: 464e11962542489482f8d1e0b4b29e09
I20260806 00:51:41.010243 5136 leader_election.cc:302] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd7245054dd14610861e2ba0e7821a48; no voters: 4c1e7f983def48678fd856d6655b41f9, ee5e9fbd481c4102aa9bcde8485eb84f
I20260806 00:51:41.010422 5745 raft_consensus.cc:2726] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:51:41.160581 5746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:41.160064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:54261)
0806 00:51:41.160410 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:41.160477 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":90,"threads_started":1}
I20260806 00:51:41.318730 5747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:41.318235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:53823)
0806 00:51:41.318567 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:41.318629 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":102,"threads_started":1}
I20260806 00:51:41.511359 5748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:41.510866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:40831)
0806 00:51:41.511194 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:41.511259 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":98,"threads_started":1}
I20260806 00:51:42.161860 5749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:42.161440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:57145)
0806 00:51:42.161686 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:42.161744 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":80,"threads_started":1}
I20260806 00:51:42.320079 5750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:42.319615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:43911)
0806 00:51:42.319900 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:42.319963 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":81,"threads_started":1}
I20260806 00:51:42.512763 5751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:42.512153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:38069)
0806 00:51:42.512597 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:42.512665 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":80,"threads_started":1}
I20260806 00:51:43.163439 5752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:43.162750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:45641)
0806 00:51:43.163257 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:43.163331 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":112,"threads_started":1}
I20260806 00:51:43.321524 5753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:43.321004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:53059)
0806 00:51:43.321348 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:43.321420 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":101,"threads_started":1}
I20260806 00:51:43.514156 5754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:43.513666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:37197)
0806 00:51:43.513989 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:43.514058 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":111,"threads_started":1}
I20260806 00:51:44.164734 5755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:44.164226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:54859)
0806 00:51:44.164532 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:44.164635 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":131,"threads_started":1}
I20260806 00:51:44.322947 5756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:44.322495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:59409)
0806 00:51:44.322782 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:44.322839 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":77,"threads_started":1}
I20260806 00:51:44.515512 5757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:44.515040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:57489)
0806 00:51:44.515338 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:44.515415 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":99,"threads_started":1}
I20260806 00:51:45.166096 5758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:45.165591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:53027)
0806 00:51:45.165908 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:45.165984 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":81,"threads_started":1}
I20260806 00:51:45.324177 5760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:45.323702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:53957)
0806 00:51:45.323996 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:45.324058 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":78,"threads_started":1}
I20260806 00:51:45.516726 5762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:45.516280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:53897)
0806 00:51:45.516578 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:45.516631 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":78,"threads_started":1}
I20260806 00:51:46.167384 5763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:46.166909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:42513)
0806 00:51:46.167195 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:46.167280 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":79,"threads_started":1}
I20260806 00:51:46.325467 5764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:46.325020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:34089)
0806 00:51:46.325292 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:46.325354 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":84,"threads_started":1}
I20260806 00:51:46.505277 5765 raft_consensus.cc:482] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:51:46.505450 5765 raft_consensus.cc:504] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee5e9fbd481c4102aa9bcde8485eb84f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 40387 } } peers { permanent_uuid: "4c1e7f983def48678fd856d6655b41f9" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38769 } } peers { permanent_uuid: "cd7245054dd14610861e2ba0e7821a48" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 42081 } }
I20260806 00:51:46.505918 5765 leader_election.cc:288] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ee5e9fbd481c4102aa9bcde8485eb84f (127.15.147.130:40387), 4c1e7f983def48678fd856d6655b41f9 (127.15.147.131:38769)
W20260806 00:51:46.506731 5136 leader_election.cc:341] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4c1e7f983def48678fd856d6655b41f9 (127.15.147.131:38769): Not found: Tablet not found: 464e11962542489482f8d1e0b4b29e09
W20260806 00:51:46.506822 5133 leader_election.cc:341] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ee5e9fbd481c4102aa9bcde8485eb84f (127.15.147.130:40387): Not found: Tablet not found: 464e11962542489482f8d1e0b4b29e09
I20260806 00:51:46.506882 5133 leader_election.cc:302] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd7245054dd14610861e2ba0e7821a48; no voters: 4c1e7f983def48678fd856d6655b41f9, ee5e9fbd481c4102aa9bcde8485eb84f
I20260806 00:51:46.507023 5765 raft_consensus.cc:2726] T 464e11962542489482f8d1e0b4b29e09 P cd7245054dd14610861e2ba0e7821a48 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:51:46.517951 5766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:46.517520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:38671)
0806 00:51:46.517789 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:46.517853 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":78,"threads_started":1}
I20260806 00:51:47.168771 5767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:47.168285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.129:51509)
0806 00:51:47.168619 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:47.168674 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":87,"threads_started":1}
I20260806 00:51:47.326783 5768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:47.326359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.130:58213)
0806 00:51:47.326622 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:47.326674 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":92,"threads_started":1}
I20260806 00:51:47.497225 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 5059
I20260806 00:51:47.510172 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 5310
I20260806 00:51:47.519248 5771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:47.518820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:40217 (local address 127.15.147.131:60261)
0806 00:51:47.519100 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:47.519155 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:40217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":90,"threads_started":1}
I20260806 00:51:47.521967 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 5493
2026-08-06T00:51:47Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19619 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:51:47 dist-test-slave-45dx krb5kdc[5774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:51:47 dist-test-slave-45dx krb5kdc[5774](info): set up 2 sockets
Aug 06 00:51:47 dist-test-slave-45dx krb5kdc[5774](info): commencing operation
krb5kdc: starting...
W20260806 00:51:48.742933 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:51:48 dist-test-slave-45dx krb5kdc[5774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:51:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:51:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:51:48.868615 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:37145
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:37145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:51:48.980079 5788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:48.980412 5788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:48.980505 5788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:48.984557 5788 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 00:51:48.984627 5788 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 00:51:48.984653 5788 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 00:51:48.984683 5788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:51:48.984724 5788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:48.984762 5788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:48.984799 5788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:51:48.984833 5788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:51:48.989845 5788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:37145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:37145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:48.990849 5788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:48.991861 5788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:49.000239 5795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:49.000245 5793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:49.000269 5799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:49.000587 5788 server_base.cc:1034] running on GCE node
I20260806 00:51:49.001248 5788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:49.002327 5788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:49.003526 5788 hybrid_clock.cc:648] HybridClock initialized: now 1785977509003501 us; error 38 us; skew 500 ppm
Aug 06 00:51:49 dist-test-slave-45dx krb5kdc[5774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977509, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:49.006389 5788 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:51:49.007305 5788 webserver.cc:460] Webserver started at http://127.15.147.190:40589/ using document root <none> and password file <none>
I20260806 00:51:49.007845 5788 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:49.007905 5788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:49.008072 5788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:51:49.010140 5788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "52d02cd310a248c6a4a40d2a82c7fe39"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "95da7076172494966d00e75f44d3454e"
server_key_iv: "25b289aa9d92aae3fa28147c75f41bac"
server_key_version: "encryptionkey@0"
I20260806 00:51:49.010668 5788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "52d02cd310a248c6a4a40d2a82c7fe39"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "95da7076172494966d00e75f44d3454e"
server_key_iv: "25b289aa9d92aae3fa28147c75f41bac"
server_key_version: "encryptionkey@0"
I20260806 00:51:49.014427 5788 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:51:49.016778 5806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:49.017947 5788 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:51:49.018076 5788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "52d02cd310a248c6a4a40d2a82c7fe39"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "95da7076172494966d00e75f44d3454e"
server_key_iv: "25b289aa9d92aae3fa28147c75f41bac"
server_key_version: "encryptionkey@0"
I20260806 00:51:49.018178 5788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:49.043841 5788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:49.047155 5788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:49.047317 5788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:49.054474 5788 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:37145
I20260806 00:51:49.054859 5878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:37145 every 8 connection(s)
I20260806 00:51:49.055511 5788 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:51:49.058297 5879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:49.063609 5879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: Bootstrap starting.
I20260806 00:51:49.065326 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 5788
I20260806 00:51:49.065466 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:51:49.065781 15950 external_mini_cluster.cc:1428] Setting key bff05a5c3d0ebebc472acd756ef96f64
I20260806 00:51:49.066533 5879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:49.068041 5879 log.cc:824] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:49.070269 5879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: No bootstrap required, opened a new log
I20260806 00:51:49.074293 5879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37145 } }
I20260806 00:51:49.074540 5879 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:51:49.074649 5879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d02cd310a248c6a4a40d2a82c7fe39, State: Initialized, Role: FOLLOWER
Aug 06 00:51:49 dist-test-slave-45dx krb5kdc[5774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:51:49.075110 5879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37145 } }
I20260806 00:51:49.075279 5879 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:51:49.075371 5879 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:51:49.075469 5879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:49.076396 5879 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37145 } }
I20260806 00:51:49.076761 5879 leader_election.cc:302] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52d02cd310a248c6a4a40d2a82c7fe39; no voters:
I20260806 00:51:49.077119 5879 leader_election.cc:288] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:51:49.077239 5887 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:51:49.077466 5887 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 1 LEADER]: Becoming Leader. State: Replica: 52d02cd310a248c6a4a40d2a82c7fe39, State: Running, Role: LEADER
I20260806 00:51:49.077780 5887 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37145 } }
I20260806 00:51:49.078465 5879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:51:49.078985 5887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52d02cd310a248c6a4a40d2a82c7fe39. Latest consensus state: current_term: 1 leader_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37145 } } }
I20260806 00:51:49.079107 5887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [sys.catalog]: This master's current role is: LEADER
I20260806 00:51:49.079770 5888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37145 } } }
I20260806 00:51:49.079864 5888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [sys.catalog]: This master's current role is: LEADER
I20260806 00:51:49.080142 5893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:51:49.082105 5884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:49.066916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37288 (local address 127.15.147.190:37145)
0806 00:51:49.067364 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:49.067387 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:49.067434 (+ 47us) server_negotiation.cc:407] Connection header received
0806 00:51:49.068271 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:49.068296 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:49.068703 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:49.069063 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:49.069739 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.071181 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.072017 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.072736 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.072925 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:49.075680 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:49.075685 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:49.075697 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:49.075723 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:49.077937 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:49.078426 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:49.078432 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:49.078437 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:49.078579 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:49.078909 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:49.078912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:49.078914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:49.079616 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:49.080662 (+ 1046us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:49.080824 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 00:51:49.081124 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":111,"threads_started":1}
I20260806 00:51:49.084226 5893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:51:49.089242 5893 catalog_manager.cc:1347] Generated new cluster ID: ea9569e939e44008b4967a519dbe8d96
I20260806 00:51:49.089305 5893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:51:49.097947 5893 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:51:49.098835 5893 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:51:49.104899 5893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: Generated new TSK 0
I20260806 00:51:49.105528 5893 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
I20260806 00:51:49.141361 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:37145
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:51:49.263833 5916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:49.264119 5916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:49.264201 5916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:49.268230 5916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:49.268308 5916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:49.268420 5916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:51:49.273526 5916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:37145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:49.274513 5916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:49.275523 5916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:49.285266 5924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:49.285327 5922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:49.285331 5921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:49.285578 5916 server_base.cc:1034] running on GCE node
I20260806 00:51:49.286024 5916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:49.286598 5916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:49.287788 5916 hybrid_clock.cc:648] HybridClock initialized: now 1785977509287764 us; error 45 us; skew 500 ppm
Aug 06 00:51:49 dist-test-slave-45dx krb5kdc[5774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977509, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:49.290612 5916 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:51:49.291451 5916 webserver.cc:460] Webserver started at http://127.15.147.129:35307/ using document root <none> and password file <none>
I20260806 00:51:49.292008 5916 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:49.292104 5916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:49.292387 5916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:51:49.294329 5916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "1986982f88094c71b5cbeb60b5e758fb"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "7f4fb4284e9769de99101ea717946bce"
server_key_iv: "bb87c240e8c04053f8937a76fab47495"
server_key_version: "encryptionkey@0"
I20260806 00:51:49.294891 5916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "1986982f88094c71b5cbeb60b5e758fb"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "7f4fb4284e9769de99101ea717946bce"
server_key_iv: "bb87c240e8c04053f8937a76fab47495"
server_key_version: "encryptionkey@0"
I20260806 00:51:49.298379 5916 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 00:51:49.300680 5933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:49.301909 5916 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:51:49.302069 5916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "1986982f88094c71b5cbeb60b5e758fb"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "7f4fb4284e9769de99101ea717946bce"
server_key_iv: "bb87c240e8c04053f8937a76fab47495"
server_key_version: "encryptionkey@0"
I20260806 00:51:49.302183 5916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:49.312228 5916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:49.315337 5916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:49.315537 5916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:49.316089 5916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:51:49.316927 5916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:51:49.317008 5916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:49.317078 5916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:51:49.317128 5916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:49.326118 5916 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:36171
I20260806 00:51:49.326143 6082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:36171 every 8 connection(s)
I20260806 00:51:49.326933 5916 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:51:49.329087 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 5916
I20260806 00:51:49.329252 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:51:49.329545 15950 external_mini_cluster.cc:1428] Setting key 55659e0264bd43f4b33a348d3dbe41e4
Aug 06 00:51:49 dist-test-slave-45dx krb5kdc[5774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977509, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.147.130@KRBTEST.COM; defaulting to no policy
I20260806 00:51:49.342700 5884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:49.328740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:58619 (local address 127.15.147.190:37145)
0806 00:51:49.328882 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:49.328887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:49.329833 (+ 946us) server_negotiation.cc:407] Connection header received
0806 00:51:49.331652 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:49.331658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:49.331740 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:49.331832 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:49.333062 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.333776 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.335348 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.335751 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.335886 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:49.338740 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:49.338743 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:49.338745 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:49.338770 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:49.340234 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:49.340912 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:49.340914 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:49.340916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:49.340993 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:49.341462 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:49.341467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:49.341469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:49.341725 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:49.341820 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:49.342404 (+ 584us) server_negotiation.cc:299] Negotiation successful
0806 00:51:49.342531 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:51:49.343441 6085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:49.329148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:58619)
0806 00:51:49.329665 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:49.329717 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:49.331447 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:49.331975 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:49.331990 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:49.332393 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:49.332894 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:49.332907 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.333907 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.333913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:49.335220 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:49.335231 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.335893 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.335896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:49.336040 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:49.336663 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:49.336679 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:49.338560 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:49.340374 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:49.340381 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:49.340398 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:49.340791 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:49.341086 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:49.341089 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:49.341091 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:49.341314 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:49.341892 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:49.341896 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:49.342110 (+ 214us) client_negotiation.cc:769] Sending connection context
0806 00:51:49.342295 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 00:51:49.342617 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":163,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.130@KRBTEST.COM" created.
I20260806 00:51:49.344753 6083 heartbeater.cc:347] Connected to a master server at 127.15.147.190:37145
I20260806 00:51:49.345016 6083 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:49.345589 6083 heartbeater.cc:511] Master 127.15.147.190:37145 requested a full tablet report, sending...
I20260806 00:51:49.347086 5830 ts_manager.cc:194] Registered new tserver with Master: 1986982f88094c71b5cbeb60b5e758fb (127.15.147.129:36171)
I20260806 00:51:49.348639 5830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:58619
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
I20260806 00:51:49.361439 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:37145
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:51:49.471884 6088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:49.472111 6088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:49.472151 6088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:49.476018 6088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:49.476080 6088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:49.476135 6088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:51:49.481285 6088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:37145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:49.482249 6088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:49.483215 6088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:49.492592 6096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:49.492597 6094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:49.492611 6098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:49.492832 6088 server_base.cc:1034] running on GCE node
I20260806 00:51:49.493252 6088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:49.493851 6088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:49.495036 6088 hybrid_clock.cc:648] HybridClock initialized: now 1785977509495029 us; error 28 us; skew 500 ppm
Aug 06 00:51:49 dist-test-slave-45dx krb5kdc[5774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977509, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:49.497794 6088 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:51:49.498529 6088 webserver.cc:460] Webserver started at http://127.15.147.130:36833/ using document root <none> and password file <none>
I20260806 00:51:49.499028 6088 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:49.499083 6088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:49.499259 6088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:51:49.501232 6088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "64dc15846a89408fab94c25dba93b2ad"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "f5992550baaa1be418eccb03866b4967"
server_key_iv: "4c2fcaed1d9918726b943feb2cb110ff"
server_key_version: "encryptionkey@0"
I20260806 00:51:49.501771 6088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "64dc15846a89408fab94c25dba93b2ad"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "f5992550baaa1be418eccb03866b4967"
server_key_iv: "4c2fcaed1d9918726b943feb2cb110ff"
server_key_version: "encryptionkey@0"
I20260806 00:51:49.505326 6088 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:51:49.507571 6111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:49.508883 6088 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:51:49.509001 6088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "64dc15846a89408fab94c25dba93b2ad"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "f5992550baaa1be418eccb03866b4967"
server_key_iv: "4c2fcaed1d9918726b943feb2cb110ff"
server_key_version: "encryptionkey@0"
I20260806 00:51:49.509102 6088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:49.558113 6088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:49.561585 6088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:49.561803 6088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:49.562446 6088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:51:49.563283 6088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:51:49.563331 6088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:49.563421 6088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:51:49.563464 6088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:49.572639 6088 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:37759
I20260806 00:51:49.572679 6272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:37759 every 8 connection(s)
I20260806 00:51:49.573513 6088 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:51:49.578972 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 6088
I20260806 00:51:49.579146 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:51:49.579504 15950 external_mini_cluster.cc:1428] Setting key dfb30f7a908031ce32c6e129ac41634d
Aug 06 00:51:49 dist-test-slave-45dx krb5kdc[5774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977509, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:51:49.588726 5884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:49.575287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:36715 (local address 127.15.147.190:37145)
0806 00:51:49.575426 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:49.575429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:49.576237 (+ 808us) server_negotiation.cc:407] Connection header received
0806 00:51:49.577664 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:49.577667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:49.577725 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:49.577871 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:49.579023 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.579844 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.581490 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.581867 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.581988 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:49.585024 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:49.585028 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:49.585031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:49.585057 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:49.586588 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:49.587133 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:49.587137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:49.587139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:49.587225 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:49.587593 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:49.587597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:49.587599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:49.587784 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:49.587897 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:49.588433 (+ 536us) server_negotiation.cc:299] Negotiation successful
0806 00:51:49.588554 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 00:51:49.589497 6276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:49.575644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:36715)
0806 00:51:49.576093 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:49.576137 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:49.577430 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:49.578048 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:49.578059 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:49.578427 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:49.578879 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:49.578892 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.579969 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.579973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:49.581338 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:49.581349 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.582082 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.582090 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:49.582349 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:49.583025 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:49.583051 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:49.584845 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:49.586718 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:49.586724 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:49.586738 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:49.587017 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:49.587322 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:49.587325 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:49.587328 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:49.587480 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:49.587905 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:49.587913 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:49.588151 (+ 238us) client_negotiation.cc:769] Sending connection context
0806 00:51:49.588352 (+ 201us) client_negotiation.cc:240] Negotiation successful
0806 00:51:49.588674 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":121,"threads_started":1}
I20260806 00:51:49.590659 6273 heartbeater.cc:347] Connected to a master server at 127.15.147.190:37145
I20260806 00:51:49.590952 6273 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.147.131@KRBTEST.COM; defaulting to no policy
I20260806 00:51:49.591526 6273 heartbeater.cc:511] Master 127.15.147.190:37145 requested a full tablet report, sending...
I20260806 00:51:49.592628 5830 ts_manager.cc:194] Registered new tserver with Master: 64dc15846a89408fab94c25dba93b2ad (127.15.147.130:37759)
I20260806 00:51:49.593242 5830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:36715
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
I20260806 00:51:49.610787 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:37145
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:51:49.724407 6281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:51:49.724659 6281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:51:49.724735 6281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:51:49.728730 6281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:51:49.728790 6281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:51:49.728883 6281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:51:49.734148 6281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:37145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:51:49.735312 6281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:51:49.736418 6281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:51:49.746644 6287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:49.746699 6288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:51:49.746699 6291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:51:49.746896 6281 server_base.cc:1034] running on GCE node
I20260806 00:51:49.747376 6281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:51:49.748073 6281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:51:49.749279 6281 hybrid_clock.cc:648] HybridClock initialized: now 1785977509749247 us; error 49 us; skew 500 ppm
Aug 06 00:51:49 dist-test-slave-45dx krb5kdc[5774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977509, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:51:49.752220 6281 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:51:49.753145 6281 webserver.cc:460] Webserver started at http://127.15.147.131:39439/ using document root <none> and password file <none>
I20260806 00:51:49.753764 6281 fs_manager.cc:359] Metadata directory not provided
I20260806 00:51:49.753865 6281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:51:49.754113 6281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:51:49.756183 6281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "3453ef5cf4c84d29b927a3eb4fd09b56"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "7280a332ab195ce1e1a8854582f4ee4b"
server_key_iv: "fdb9a9008c9506cc829d14e4ff568d22"
server_key_version: "encryptionkey@0"
I20260806 00:51:49.756819 6281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "3453ef5cf4c84d29b927a3eb4fd09b56"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "7280a332ab195ce1e1a8854582f4ee4b"
server_key_iv: "fdb9a9008c9506cc829d14e4ff568d22"
server_key_version: "encryptionkey@0"
I20260806 00:51:49.760483 6281 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:51:49.762817 6300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:49.763957 6281 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:51:49.764088 6281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "3453ef5cf4c84d29b927a3eb4fd09b56"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "7280a332ab195ce1e1a8854582f4ee4b"
server_key_iv: "fdb9a9008c9506cc829d14e4ff568d22"
server_key_version: "encryptionkey@0"
I20260806 00:51:49.764197 6281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:51:49.776281 6281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:51:49.779820 6281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:51:49.780014 6281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:51:49.780717 6281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:51:49.781539 6281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:51:49.781611 6281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:49.781677 6281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:51:49.781726 6281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:51:49.791231 6281 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:45087
I20260806 00:51:49.791266 6465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:45087 every 8 connection(s)
I20260806 00:51:49.792240 6281 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:51:49.797338 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 6281
I20260806 00:51:49.797459 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:51:49.797792 15950 external_mini_cluster.cc:1428] Setting key 58aa8918813376cbcb82af6fa8dec461
Aug 06 00:51:49 dist-test-slave-45dx krb5kdc[5774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977509, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:51:49.805612 5884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:49.794189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:46441 (local address 127.15.147.190:37145)
0806 00:51:49.794337 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:49.794342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:49.795111 (+ 769us) server_negotiation.cc:407] Connection header received
0806 00:51:49.796440 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:49.796442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:49.796491 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:49.796572 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:49.797761 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.798338 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.799468 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.799802 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.799901 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:49.802215 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:49.802218 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:49.802220 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:49.802245 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:49.803729 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:49.804178 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:49.804180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:49.804181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:49.804262 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:49.804648 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:49.804650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:49.804652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:49.804818 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:49.804896 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:49.805401 (+ 505us) server_negotiation.cc:299] Negotiation successful
0806 00:51:49.805497 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:51:49.806469 6470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:49.794540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:46441)
0806 00:51:49.794960 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:49.795008 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:49.796180 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:49.796727 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:49.796733 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:49.797062 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:51:49.797623 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:49.797639 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.798469 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.798472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:49.799338 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:49.799347 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.799941 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.799945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:49.800123 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:49.800624 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:51:49.800640 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:51:49.802065 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:51:49.803833 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:49.803837 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:49.803847 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:49.804092 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:49.804405 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:51:49.804407 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:51:49.804409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:51:49.804552 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:51:49.804921 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:51:49.804926 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:51:49.805126 (+ 200us) client_negotiation.cc:769] Sending connection context
0806 00:51:49.805338 (+ 212us) client_negotiation.cc:240] Negotiation successful
0806 00:51:49.805634 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":109,"threads_started":1}
I20260806 00:51:49.807582 6466 heartbeater.cc:347] Connected to a master server at 127.15.147.190:37145
I20260806 00:51:49.807811 6466 heartbeater.cc:464] Registering TS with master...
I20260806 00:51:49.808370 6466 heartbeater.cc:511] Master 127.15.147.190:37145 requested a full tablet report, sending...
I20260806 00:51:49.809425 5830 ts_manager.cc:194] Registered new tserver with Master: 3453ef5cf4c84d29b927a3eb4fd09b56 (127.15.147.131:45087)
I20260806 00:51:49.810037 5830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:46441
I20260806 00:51:49.811482 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:51:49.820217 5884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:49.813621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37294 (local address 127.15.147.190:37145)
0806 00:51:49.813727 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:49.813731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:49.813949 (+ 218us) server_negotiation.cc:407] Connection header received
0806 00:51:49.814030 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:49.814032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:49.814066 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:49.814124 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:51:49.814744 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.815194 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.815953 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.816142 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.816211 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:49.817446 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:51:49.817449 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:51:49.817451 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:51:49.817471 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:51:49.818895 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:49.819260 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:49.819263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:49.819264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:49.819337 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:51:49.819562 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:51:49.819564 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:51:49.819565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:51:49.819755 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:51:49.819856 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:49.820027 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 00:51:49.820112 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:51:49.822957 5830 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:51:49.826179 5830 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:51:49.840876 6487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:49.835239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:45087 (local address 127.0.0.1:43682)
0806 00:51:49.836251 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:49.836265 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:49.836402 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:49.836878 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:49.836881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:49.836898 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:49.837008 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:49.837016 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.838161 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.838164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:49.839233 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:49.839244 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.840265 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.840269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:49.840425 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:49.840470 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 00:51:49.840592 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 00:51:49.840724 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"mutex_wait_us":36,"thread_start_us":89,"threads_started":1}
I20260806 00:51:49.841355 6486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:49.835350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:36171 (local address 127.0.0.1:44746)
0806 00:51:49.836537 (+ 1187us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:49.836551 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:49.836649 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:49.837140 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:49.837145 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:49.837165 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:49.837245 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:49.837250 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.838646 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.838650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:49.839765 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:49.839773 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.840944 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.840947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:49.841037 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:49.841059 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:51:49.841132 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 00:51:49.841211 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1125,"thread_start_us":110,"threads_started":1}
I20260806 00:51:49.841357 6482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:49.834477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:37759 (local address 127.0.0.1:42306)
0806 00:51:49.835310 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:49.835362 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:49.835503 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:49.836414 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:49.836422 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:49.836446 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:49.836577 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:49.836582 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.838092 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.838096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:49.839571 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:49.839583 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.840884 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.840888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:49.840989 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:49.841023 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 00:51:49.841097 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 00:51:49.841165 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":63,"thread_start_us":95,"threads_started":1}
I20260806 00:51:49.841594 6484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:49.834635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42306 (local address 127.15.147.130:37759)
0806 00:51:49.835011 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:49.835024 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:49.835653 (+ 629us) server_negotiation.cc:407] Connection header received
0806 00:51:49.835747 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:49.835752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:49.835983 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:49.836155 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:49.836731 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.837932 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.839709 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.840577 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.840702 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:49.840817 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:49.841295 (+ 478us) server_negotiation.cc:299] Negotiation successful
0806 00:51:49.841398 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":151,"threads_started":1}
I20260806 00:51:49.840922 6488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:49.835407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43682 (local address 127.15.147.131:45087)
0806 00:51:49.836043 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:49.836068 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:49.836364 (+ 296us) server_negotiation.cc:407] Connection header received
0806 00:51:49.836547 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:49.836553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:49.836751 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:49.836905 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:49.837108 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.838002 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.839363 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.840130 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.840257 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:49.840430 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:49.840646 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 00:51:49.840752 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":161,"threads_started":1}
I20260806 00:51:49.842170 6489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:49.835815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44746 (local address 127.15.147.129:36171)
0806 00:51:49.836137 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:49.836149 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:49.836661 (+ 512us) server_negotiation.cc:407] Connection header received
0806 00:51:49.836747 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:49.836751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:49.836937 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:49.837083 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:49.837369 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.838467 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.839882 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:49.840637 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:49.840747 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:49.840864 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:49.841912 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0806 00:51:49.842012 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":103,"threads_started":1}
I20260806 00:51:49.844475 6178 tablet_service.cc:1467] Processing CreateTablet for tablet c61d376d7e4d48eaa942cef26329fb00 (DEFAULT_TABLE table=test-table [id=783bb810234543e490cdca2719212242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:51:49.844700 5991 tablet_service.cc:1467] Processing CreateTablet for tablet c61d376d7e4d48eaa942cef26329fb00 (DEFAULT_TABLE table=test-table [id=783bb810234543e490cdca2719212242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:51:49.844707 6373 tablet_service.cc:1467] Processing CreateTablet for tablet c61d376d7e4d48eaa942cef26329fb00 (DEFAULT_TABLE table=test-table [id=783bb810234543e490cdca2719212242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:51:49.845575 6178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c61d376d7e4d48eaa942cef26329fb00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:49.845715 6373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c61d376d7e4d48eaa942cef26329fb00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:49.845707 5991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c61d376d7e4d48eaa942cef26329fb00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:51:49.851852 6491 tablet_bootstrap.cc:492] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad: Bootstrap starting.
I20260806 00:51:49.853278 6493 tablet_bootstrap.cc:492] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56: Bootstrap starting.
I20260806 00:51:49.853379 6492 tablet_bootstrap.cc:492] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb: Bootstrap starting.
I20260806 00:51:49.854467 6491 tablet_bootstrap.cc:654] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:49.855242 6491 log.cc:824] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:49.855412 6492 tablet_bootstrap.cc:654] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:49.855412 6493 tablet_bootstrap.cc:654] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56: Neither blocks nor log segments found. Creating new log.
I20260806 00:51:49.856333 6492 log.cc:824] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:49.856707 6493 log.cc:824] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56: Log is configured to *not* fsync() on all Append() calls
I20260806 00:51:49.857782 6491 tablet_bootstrap.cc:492] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad: No bootstrap required, opened a new log
I20260806 00:51:49.857980 6491 ts_tablet_manager.cc:1397] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:51:49.858270 6492 tablet_bootstrap.cc:492] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb: No bootstrap required, opened a new log
I20260806 00:51:49.858465 6492 ts_tablet_manager.cc:1397] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:51:49.858702 6493 tablet_bootstrap.cc:492] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56: No bootstrap required, opened a new log
I20260806 00:51:49.858898 6493 ts_tablet_manager.cc:1397] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:51:49.861842 6492 raft_consensus.cc:348] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:51:49.862083 6492 raft_consensus.cc:374] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:51:49.862150 6492 raft_consensus.cc:729] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1986982f88094c71b5cbeb60b5e758fb, State: Initialized, Role: FOLLOWER
I20260806 00:51:49.861971 6491 raft_consensus.cc:348] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:51:49.862427 6491 raft_consensus.cc:374] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:51:49.862484 6491 raft_consensus.cc:729] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64dc15846a89408fab94c25dba93b2ad, State: Initialized, Role: FOLLOWER
I20260806 00:51:49.862656 6492 consensus_queue.cc:260] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:51:49.862654 6493 raft_consensus.cc:348] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:51:49.862902 6493 raft_consensus.cc:374] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:51:49.862958 6493 raft_consensus.cc:729] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3453ef5cf4c84d29b927a3eb4fd09b56, State: Initialized, Role: FOLLOWER
I20260806 00:51:49.862919 6491 consensus_queue.cc:260] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:51:49.863376 6493 consensus_queue.cc:260] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:51:49.863577 6273 heartbeater.cc:503] Master 127.15.147.190:37145 was elected leader, sending a full tablet report...
I20260806 00:51:49.863759 6492 ts_tablet_manager.cc:1428] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 00:51:49.863883 6083 heartbeater.cc:503] Master 127.15.147.190:37145 was elected leader, sending a full tablet report...
I20260806 00:51:49.864279 6493 ts_tablet_manager.cc:1428] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 00:51:49.864746 6466 heartbeater.cc:503] Master 127.15.147.190:37145 was elected leader, sending a full tablet report...
I20260806 00:51:49.864964 6491 ts_tablet_manager.cc:1428] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260806 00:51:50.043272 6469 tablet.cc:2369] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:51:50.074798 6274 tablet.cc:2369] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:51:50.078250 6084 tablet.cc:2369] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:51:50.222415 6500 raft_consensus.cc:482] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:51:50.222697 6500 raft_consensus.cc:504] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:51:50.223906 6500 leader_election.cc:288] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64dc15846a89408fab94c25dba93b2ad (127.15.147.130:37759), 3453ef5cf4c84d29b927a3eb4fd09b56 (127.15.147.131:45087)
I20260806 00:51:50.228312 6505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:50.224245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:37759 (local address 127.15.147.129:43551)
0806 00:51:50.224579 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:50.224594 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:50.224725 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:50.225061 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:50.225064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:50.225085 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:50.225212 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:50.225218 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:50.226159 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:50.226163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:50.227088 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:50.227098 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:50.227796 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:50.227799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:50.227917 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:50.227949 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 00:51:50.228035 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 00:51:50.228122 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":161,"threads_started":1}
I20260806 00:51:50.228320 6484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:50.224349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:43551 (local address 127.15.147.130:37759)
0806 00:51:50.224513 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:50.224517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:50.224702 (+ 185us) server_negotiation.cc:407] Connection header received
0806 00:51:50.224833 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:50.224836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:50.224893 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:50.224991 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:50.225376 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:50.226045 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:50.227224 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:50.227677 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:50.227766 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:50.227805 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:50.228074 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 00:51:50.228164 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 00:51:50.228489 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:50.224734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:45087 (local address 127.15.147.129:60111)
0806 00:51:50.225185 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:50.225196 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:51:50.225280 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:51:50.225551 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:51:50.225553 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:51:50.225573 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:51:50.225701 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:50.225706 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:50.226500 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:50.226502 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:50.227272 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:51:50.227282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:51:50.227983 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:51:50.227986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:51:50.228133 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:50.228166 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 00:51:50.228241 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 00:51:50.228344 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":127,"threads_started":1}
I20260806 00:51:50.228564 6488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:50.224909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:60111 (local address 127.15.147.131:45087)
0806 00:51:50.225080 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:50.225085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:50.225285 (+ 200us) server_negotiation.cc:407] Connection header received
0806 00:51:50.225398 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:50.225400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:50.225440 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:50.225495 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:51:50.225807 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:50.226369 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:50.227407 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:50.227856 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:50.227968 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:50.228048 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:50.228278 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 00:51:50.228407 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 00:51:50.229156 6207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c61d376d7e4d48eaa942cef26329fb00" candidate_uuid: "1986982f88094c71b5cbeb60b5e758fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64dc15846a89408fab94c25dba93b2ad" is_pre_election: true
I20260806 00:51:50.229151 6396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c61d376d7e4d48eaa942cef26329fb00" candidate_uuid: "1986982f88094c71b5cbeb60b5e758fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" is_pre_election: true
I20260806 00:51:50.229481 6396 raft_consensus.cc:2454] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1986982f88094c71b5cbeb60b5e758fb in term 0.
I20260806 00:51:50.229485 6207 raft_consensus.cc:2454] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1986982f88094c71b5cbeb60b5e758fb in term 0.
I20260806 00:51:50.229995 5934 leader_election.cc:302] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1986982f88094c71b5cbeb60b5e758fb, 3453ef5cf4c84d29b927a3eb4fd09b56; no voters:
I20260806 00:51:50.230269 6500 raft_consensus.cc:2781] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:51:50.230351 6500 raft_consensus.cc:482] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:51:50.230382 6500 raft_consensus.cc:3037] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:50.231475 6500 raft_consensus.cc:504] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:51:50.231848 6500 leader_election.cc:288] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 1 election: Requested vote from peers 64dc15846a89408fab94c25dba93b2ad (127.15.147.130:37759), 3453ef5cf4c84d29b927a3eb4fd09b56 (127.15.147.131:45087)
I20260806 00:51:50.232223 6207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c61d376d7e4d48eaa942cef26329fb00" candidate_uuid: "1986982f88094c71b5cbeb60b5e758fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64dc15846a89408fab94c25dba93b2ad"
I20260806 00:51:50.232347 6396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c61d376d7e4d48eaa942cef26329fb00" candidate_uuid: "1986982f88094c71b5cbeb60b5e758fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56"
I20260806 00:51:50.232375 6207 raft_consensus.cc:3037] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:50.232492 6396 raft_consensus.cc:3037] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:51:50.233506 6207 raft_consensus.cc:2454] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1986982f88094c71b5cbeb60b5e758fb in term 1.
I20260806 00:51:50.233676 6396 raft_consensus.cc:2454] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1986982f88094c71b5cbeb60b5e758fb in term 1.
I20260806 00:51:50.233852 5934 leader_election.cc:302] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1986982f88094c71b5cbeb60b5e758fb, 64dc15846a89408fab94c25dba93b2ad; no voters:
I20260806 00:51:50.234041 6500 raft_consensus.cc:2781] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:51:50.234259 6500 raft_consensus.cc:686] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [term 1 LEADER]: Becoming Leader. State: Replica: 1986982f88094c71b5cbeb60b5e758fb, State: Running, Role: LEADER
I20260806 00:51:50.234612 6500 consensus_queue.cc:237] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:51:50.237991 5830 catalog_manager.cc:5495] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 1986982f88094c71b5cbeb60b5e758fb (127.15.147.129). New cstate: current_term: 1 leader_uuid: "1986982f88094c71b5cbeb60b5e758fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } health_report { overall_health: HEALTHY } } }
I20260806 00:51:50.289800 6489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:50.286488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44760 (local address 127.15.147.129:36171)
0806 00:51:50.286631 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 00:51:50.286635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:51:50.286647 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:51:50.286722 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:51:50.286724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:51:50.286776 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:51:50.286856 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:51:50.287081 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:50.287611 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:50.288455 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:51:50.288684 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:51:50.288760 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:51:50.289031 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:51:50.289444 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:51:50.289543 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 00:51:50.289599 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 00:51:50.289648 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:51:50.295892 6207 raft_consensus.cc:1261] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad [term 1 FOLLOWER]: Refusing update from remote peer 1986982f88094c71b5cbeb60b5e758fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:51:50.295892 6396 raft_consensus.cc:1261] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 1 FOLLOWER]: Refusing update from remote peer 1986982f88094c71b5cbeb60b5e758fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:51:50.296670 6500 consensus_queue.cc:1035] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:51:50.296912 6509 consensus_queue.cc:1035] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:51:50.303906 6512 mvcc.cc:205] Tried to move back new op lower bound from 7315363882163478528 to 7315363881923534848. Current Snapshot: MvccSnapshot[applied={T|T < 7315363882163478528}]
I20260806 00:51:50.304800 6514 mvcc.cc:205] Tried to move back new op lower bound from 7315363882163478528 to 7315363881923534848. Current Snapshot: MvccSnapshot[applied={T|T < 7315363882163478528}]
I20260806 00:51:50.313488 5828 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:51:50.313627 5828 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:51:50.316022 5828 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: Sending DeleteTablet for 3 replicas of tablet c61d376d7e4d48eaa942cef26329fb00
I20260806 00:51:50.316915 6373 tablet_service.cc:1514] Processing DeleteTablet for tablet c61d376d7e4d48eaa942cef26329fb00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:51:50 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:43682
I20260806 00:51:50.316915 6178 tablet_service.cc:1514] Processing DeleteTablet for tablet c61d376d7e4d48eaa942cef26329fb00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:51:50 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:42306
I20260806 00:51:50.317389 6530 tablet_replica.cc:330] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56: stopping tablet replica
I20260806 00:51:50.317399 5991 tablet_service.cc:1514] Processing DeleteTablet for tablet c61d376d7e4d48eaa942cef26329fb00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:51:50 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:44746
I20260806 00:51:50.317516 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 5916
I20260806 00:51:50.319865 6531 tablet_replica.cc:330] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad: stopping tablet replica
I20260806 00:51:50.320281 6531 raft_consensus.cc:2229] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:51:50.320703 6531 raft_consensus.cc:2258] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:51:50.324190 6530 raft_consensus.cc:2229] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:51:50.324575 6530 raft_consensus.cc:2258] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:51:50.327657 6530 ts_tablet_manager.cc:1905] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:51:50.329119 6531 ts_tablet_manager.cc:1905] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:51:50.330606 6530 ts_tablet_manager.cc:1918] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:51:50.330727 6530 log.cc:1196] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/c61d376d7e4d48eaa942cef26329fb00
I20260806 00:51:50.331105 6530 ts_tablet_manager.cc:1939] T c61d376d7e4d48eaa942cef26329fb00 P 3453ef5cf4c84d29b927a3eb4fd09b56: Deleting consensus metadata
I20260806 00:51:50.332082 6531 ts_tablet_manager.cc:1918] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:51:50.332321 6531 log.cc:1196] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/c61d376d7e4d48eaa942cef26329fb00
I20260806 00:51:50.332583 5807 catalog_manager.cc:4841] TS 3453ef5cf4c84d29b927a3eb4fd09b56 (127.15.147.131:45087): tablet c61d376d7e4d48eaa942cef26329fb00 (table test-table [id=783bb810234543e490cdca2719212242]) successfully deleted
I20260806 00:51:50.332715 6531 ts_tablet_manager.cc:1939] T c61d376d7e4d48eaa942cef26329fb00 P 64dc15846a89408fab94c25dba93b2ad: Deleting consensus metadata
I20260806 00:51:50.333787 5807 catalog_manager.cc:4841] TS 64dc15846a89408fab94c25dba93b2ad (127.15.147.130:37759): tablet c61d376d7e4d48eaa942cef26329fb00 (table test-table [id=783bb810234543e490cdca2719212242]) successfully deleted
W20260806 00:51:50.335327 5808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.147.129:36171 (error 108)
I20260806 00:51:50.335628 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 6088
I20260806 00:51:50.338048 6486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:51:50.337768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:36171 (local address 127.0.0.1:44776)
0806 00:51:50.337906 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 00:51:50.337974 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:36171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 00:51:50.338311 5808 catalog_manager.cc:4584] TS 1986982f88094c71b5cbeb60b5e758fb (127.15.147.129:36171): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c61d376d7e4d48eaa942cef26329fb00: Network error: Client connection negotiation failed: client connection to 127.15.147.129:36171: connect: Connection refused (error 111)
I20260806 00:51:50.347656 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 6281
I20260806 00:51:50.358229 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 5788
I20260806 00:52:10.370990 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:37145
--webserver_interface=127.15.147.190
--webserver_port=40589
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:37145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:52:10.487640 6534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:10.487890 6534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:10.487927 6534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:10.491750 6534 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 00:52:10.491801 6534 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 00:52:10.491816 6534 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 00:52:10.491829 6534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:52:10.491847 6534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:10.491865 6534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:10.491881 6534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:52:10.491895 6534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:52:10.497061 6534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:37145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:37145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=40589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:10.498035 6534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:10.498957 6534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:10.513924 6541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:10.513969 6543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:10.513985 6540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:10.514140 6534 server_base.cc:1034] running on GCE node
I20260806 00:52:10.514822 6534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:10.516033 6534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:10.517223 6534 hybrid_clock.cc:648] HybridClock initialized: now 1785977530517202 us; error 31 us; skew 500 ppm
Aug 06 00:52:10 dist-test-slave-45dx krb5kdc[5774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977530, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:10.524791 6534 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:52:10.525715 6534 webserver.cc:460] Webserver started at http://127.15.147.190:40589/ using document root <none> and password file <none>
I20260806 00:52:10.526314 6534 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:10.526393 6534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:10.530421 6534 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:52:10.532646 6553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:10.533902 6534 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:52:10.534044 6534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "52d02cd310a248c6a4a40d2a82c7fe39"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "95da7076172494966d00e75f44d3454e"
server_key_iv: "25b289aa9d92aae3fa28147c75f41bac"
server_key_version: "encryptionkey@0"
I20260806 00:52:10.534641 6534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:10.546465 6534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:10.549239 6534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:10.549464 6534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:10.556576 6534 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:37145
I20260806 00:52:10.556908 6627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:37145 every 8 connection(s)
I20260806 00:52:10.557667 6534 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:52:10.559621 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 6534
I20260806 00:52:10.560120 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:36171
--local_ip_for_outbound_sockets=127.15.147.129
--tserver_master_addrs=127.15.147.190:37145
--webserver_port=35307
--webserver_interface=127.15.147.129
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:52:10.562810 6628 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:52:10.564571 6628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: Bootstrap starting.
I20260806 00:52:10.582751 6628 log.cc:824] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:10.591526 6628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:52:10.591922 6628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: Bootstrap complete.
I20260806 00:52:10.596072 6628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37145 } }
I20260806 00:52:10.596570 6628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d02cd310a248c6a4a40d2a82c7fe39, State: Initialized, Role: FOLLOWER
I20260806 00:52:10.597091 6628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37145 } }
I20260806 00:52:10.597316 6628 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:52:10.597430 6628 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:52:10.597549 6628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 1 FOLLOWER]: Advancing to term 2
I20260806 00:52:10.599364 6628 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37145 } }
I20260806 00:52:10.599709 6628 leader_election.cc:302] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52d02cd310a248c6a4a40d2a82c7fe39; no voters:
I20260806 00:52:10.600013 6628 leader_election.cc:288] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 00:52:10.600167 6633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 00:52:10.600504 6633 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 2 LEADER]: Becoming Leader. State: Replica: 52d02cd310a248c6a4a40d2a82c7fe39, State: Running, Role: LEADER
I20260806 00:52:10.600852 6633 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37145 } }
I20260806 00:52:10.601257 6628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:52:10.601964 6633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52d02cd310a248c6a4a40d2a82c7fe39. Latest consensus state: current_term: 2 leader_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37145 } } }
I20260806 00:52:10.602111 6633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [sys.catalog]: This master's current role is: LEADER
I20260806 00:52:10.602514 6645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:52:10.602540 6634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37145 } } }
I20260806 00:52:10.602757 6634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [sys.catalog]: This master's current role is: LEADER
I20260806 00:52:10.607616 6645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:52:10.608536 6645 catalog_manager.cc:1259] Loaded cluster ID: ea9569e939e44008b4967a519dbe8d96
I20260806 00:52:10.608604 6645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:52:10.611421 6645 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:52:10.612422 6645 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: Loaded TSK: 0
I20260806 00:52:10.629103 6645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: Generated new TSK 1
I20260806 00:52:10.629374 6645 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:52:10.695533 6630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:10.695794 6630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:10.695842 6630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:10.700501 6630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:10.700560 6630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:10.700656 6630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:52:10.706689 6630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:36171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=35307
--tserver_master_addrs=127.15.147.190:37145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:10.707736 6630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:10.708863 6630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:10.719960 6669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:10.719970 6670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:10.720045 6673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:10.720525 6630 server_base.cc:1034] running on GCE node
I20260806 00:52:10.720965 6630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:10.721592 6630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:10.722800 6630 hybrid_clock.cc:648] HybridClock initialized: now 1785977530722776 us; error 43 us; skew 500 ppm
Aug 06 00:52:10 dist-test-slave-45dx krb5kdc[5774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977530, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:10.725656 6630 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:52:10.726617 6630 webserver.cc:460] Webserver started at http://127.15.147.129:35307/ using document root <none> and password file <none>
I20260806 00:52:10.727180 6630 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:10.727296 6630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:10.731215 6630 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:52:10.733428 6689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:10.734704 6630 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:52:10.734833 6630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "1986982f88094c71b5cbeb60b5e758fb"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "7f4fb4284e9769de99101ea717946bce"
server_key_iv: "bb87c240e8c04053f8937a76fab47495"
server_key_version: "encryptionkey@0"
I20260806 00:52:10.735373 6630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:10.748083 6630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:10.751114 6630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:10.751322 6630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:10.751874 6630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:10.752970 6696 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 00:52:10.757018 6630 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 00:52:10.757077 6630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260806 00:52:10.757150 6630 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 00:52:10.758773 6630 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 00:52:10.758833 6630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 00:52:10.759085 6696 tablet_bootstrap.cc:492] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb: Bootstrap starting.
I20260806 00:52:10.770386 6630 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:36171
I20260806 00:52:10.771600 6630 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:52:10.772495 6842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:36171 every 8 connection(s)
I20260806 00:52:10.778792 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 6630
I20260806 00:52:10.779376 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:37759
--local_ip_for_outbound_sockets=127.15.147.130
--tserver_master_addrs=127.15.147.190:37145
--webserver_port=36833
--webserver_interface=127.15.147.130
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:52:10.782735 6696 log.cc:824] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:52:10 dist-test-slave-45dx krb5kdc[5774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977530, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:52:10.791038 6696 tablet_bootstrap.cc:492] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:52:10.791505 6696 tablet_bootstrap.cc:492] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb: Bootstrap complete.
I20260806 00:52:10.792878 6847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:10.775822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:37501 (local address 127.15.147.190:37145)
0806 00:52:10.776723 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:10.776744 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:10.776775 (+ 31us) server_negotiation.cc:407] Connection header received
0806 00:52:10.776977 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:10.776987 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:10.777251 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:10.777588 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:10.779128 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:10.780926 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:10.782597 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:10.783347 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:10.783538 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:10.786679 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:10.786685 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:10.786699 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:10.786730 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:10.788914 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:10.789638 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:10.789645 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:10.789650 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:10.789811 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:10.790258 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:10.790261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:10.790263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:10.790699 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:10.790838 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:10.791630 (+ 792us) server_negotiation.cc:299] Negotiation successful
0806 00:52:10.791945 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":124,"threads_started":1}
I20260806 00:52:10.792932 6846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:10.774440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:37501)
0806 00:52:10.775077 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:10.775147 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:10.776361 (+ 1214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:10.777635 (+ 1274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:10.777643 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:10.778225 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:10.778840 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:10.778858 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:10.781093 (+ 2235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:10.781103 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:10.782450 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:10.782462 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:10.783516 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:10.783520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:10.783734 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:10.784462 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:52:10.784481 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:10.786452 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:10.789049 (+ 2597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:10.789057 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:10.789072 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:10.789499 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:10.789929 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:10.789935 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:10.789938 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:10.790139 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:10.790837 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:10.790846 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:10.791199 (+ 353us) client_negotiation.cc:769] Sending connection context
0806 00:52:10.791436 (+ 237us) client_negotiation.cc:240] Negotiation successful
0806 00:52:10.791813 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":150,"threads_started":1}
I20260806 00:52:10.793157 6696 ts_tablet_manager.cc:1397] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb: Time spent bootstrapping tablet: real 0.034s user 0.018s sys 0.015s
I20260806 00:52:10.795825 6844 heartbeater.cc:347] Connected to a master server at 127.15.147.190:37145
I20260806 00:52:10.796062 6844 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:10.795640 6696 raft_consensus.cc:348] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:52:10.796173 6696 raft_consensus.cc:729] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1986982f88094c71b5cbeb60b5e758fb, State: Initialized, Role: FOLLOWER
I20260806 00:52:10.796612 6844 heartbeater.cc:511] Master 127.15.147.190:37145 requested a full tablet report, sending...
I20260806 00:52:10.796705 6696 consensus_queue.cc:260] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:52:10.797780 6696 ts_tablet_manager.cc:1428] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:52:10.799129 6575 ts_manager.cc:194] Registered new tserver with Master: 1986982f88094c71b5cbeb60b5e758fb (127.15.147.129:36171)
I20260806 00:52:10.802591 6575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:37501
I20260806 00:52:10.805219 6844 heartbeater.cc:503] Master 127.15.147.190:37145 was elected leader, sending a full tablet report...
I20260806 00:52:10.808416 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:10.801938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:36171 (local address 127.0.0.1:44202)
0806 00:52:10.802750 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:10.802792 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:10.802920 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:10.804549 (+ 1629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:10.804555 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:10.804578 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:10.804699 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:10.804705 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:10.805741 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:10.805744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:10.807033 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:10.807042 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:10.807830 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:10.807833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:10.807942 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:10.807980 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 00:52:10.808113 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 00:52:10.808231 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":96,"threads_started":1}
I20260806 00:52:10.808415 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:10.802020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44202 (local address 127.15.147.129:36171)
0806 00:52:10.803784 (+ 1764us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:10.803793 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:10.803809 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:52:10.803860 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:10.803864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:10.804360 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:10.804547 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:10.804814 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:10.805650 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:10.807179 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:10.807678 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:10.807839 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:10.807970 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:10.808140 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 00:52:10.808226 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1688,"thread_start_us":103,"threads_started":1}
I20260806 00:52:10.809427 6752 tablet_service.cc:1514] Processing DeleteTablet for tablet c61d376d7e4d48eaa942cef26329fb00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:51:50 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:44202
I20260806 00:52:10.810173 6858 tablet_replica.cc:330] stopping tablet replica
I20260806 00:52:10.810457 6858 raft_consensus.cc:2229] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:52:10.810693 6858 raft_consensus.cc:2258] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:10.811684 6858 ts_tablet_manager.cc:1905] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:52:10.815284 6858 ts_tablet_manager.cc:1918] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:52:10.815366 6858 log.cc:1196] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/wal/wals/c61d376d7e4d48eaa942cef26329fb00
I20260806 00:52:10.815634 6858 ts_tablet_manager.cc:1939] T c61d376d7e4d48eaa942cef26329fb00 P 1986982f88094c71b5cbeb60b5e758fb: Deleting consensus metadata
I20260806 00:52:10.817313 6555 catalog_manager.cc:4841] TS 1986982f88094c71b5cbeb60b5e758fb (127.15.147.129:36171): tablet c61d376d7e4d48eaa942cef26329fb00 (table test-table [id=783bb810234543e490cdca2719212242]) successfully deleted
W20260806 00:52:10.900282 6850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:10.900604 6850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:10.900687 6850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:10.905033 6850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:10.905107 6850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:10.905196 6850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:52:10.910421 6850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:37759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=36833
--tserver_master_addrs=127.15.147.190:37145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:10.911463 6850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:10.912557 6850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:10.922358 6866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:10.922372 6865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:10.922372 6868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:10.922442 6850 server_base.cc:1034] running on GCE node
I20260806 00:52:10.923041 6850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:10.923637 6850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:10.924821 6850 hybrid_clock.cc:648] HybridClock initialized: now 1785977530924806 us; error 27 us; skew 500 ppm
Aug 06 00:52:10 dist-test-slave-45dx krb5kdc[5774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977530, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:10.927548 6850 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:52:10.928525 6850 webserver.cc:460] Webserver started at http://127.15.147.130:36833/ using document root <none> and password file <none>
I20260806 00:52:10.929070 6850 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:10.929178 6850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:10.932837 6850 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 00:52:10.934762 6876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:10.935891 6850 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:52:10.936035 6850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "64dc15846a89408fab94c25dba93b2ad"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "f5992550baaa1be418eccb03866b4967"
server_key_iv: "4c2fcaed1d9918726b943feb2cb110ff"
server_key_version: "encryptionkey@0"
I20260806 00:52:10.936637 6850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:10.953038 6850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:10.955896 6850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:10.956120 6850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:10.956787 6850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:10.957616 6850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:10.957719 6850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:10.957788 6850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:10.957844 6850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:10.967036 6850 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:37759
I20260806 00:52:10.967089 7042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:37759 every 8 connection(s)
I20260806 00:52:10.968017 6850 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:52:10.969630 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 6850
I20260806 00:52:10.970147 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:45087
--local_ip_for_outbound_sockets=127.15.147.131
--tserver_master_addrs=127.15.147.190:37145
--webserver_port=39439
--webserver_interface=127.15.147.131
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 00:52:10 dist-test-slave-45dx krb5kdc[5774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977530, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:52:10.983604 6847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:10.969888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:47025 (local address 127.15.147.190:37145)
0806 00:52:10.970054 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:10.970059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:10.971130 (+ 1071us) server_negotiation.cc:407] Connection header received
0806 00:52:10.972865 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:10.972869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:10.972943 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:10.973053 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:10.974197 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:10.975045 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:10.976197 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:10.976661 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:10.976803 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:10.979499 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:10.979505 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:10.979510 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:10.979544 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:10.981367 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:10.981899 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:10.981905 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:10.981910 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:10.982017 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:10.982358 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:10.982364 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:10.982367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:10.982631 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:10.982748 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:10.983212 (+ 464us) server_negotiation.cc:299] Negotiation successful
0806 00:52:10.983368 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 00:52:10.984247 7045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:10.970374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:47025)
0806 00:52:10.970934 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:10.970995 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:10.972645 (+ 1650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:10.973230 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:10.973237 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:10.973591 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:10.974038 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:10.974052 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:10.975197 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:10.975200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:10.976067 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:10.976077 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:10.976835 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:10.976838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:10.976991 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:10.977628 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:52:10.977643 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:10.979299 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:10.981490 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:10.981494 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:10.981505 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:10.981780 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:10.982117 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:10.982119 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:10.982120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:10.982254 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:10.982751 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:10.982757 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:10.982969 (+ 212us) client_negotiation.cc:769] Sending connection context
0806 00:52:10.983116 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 00:52:10.983358 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":114,"threads_started":1}
I20260806 00:52:10.985577 7043 heartbeater.cc:347] Connected to a master server at 127.15.147.190:37145
I20260806 00:52:10.985817 7043 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:10.986317 7043 heartbeater.cc:511] Master 127.15.147.190:37145 requested a full tablet report, sending...
I20260806 00:52:10.987447 6575 ts_manager.cc:194] Registered new tserver with Master: 64dc15846a89408fab94c25dba93b2ad (127.15.147.130:37759)
I20260806 00:52:10.988185 6575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:47025
W20260806 00:52:11.088851 7046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:11.089093 7046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:11.089155 7046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:11.093122 7046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:11.093173 7046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:11.093236 7046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:52:11.098138 7046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:45087
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=39439
--tserver_master_addrs=127.15.147.190:37145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:11.099144 7046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:11.100136 7046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:11.109876 7054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:11.109926 7056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:11.109926 7059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:11.110069 7046 server_base.cc:1034] running on GCE node
I20260806 00:52:11.110452 7046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:11.111105 7046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:11.112337 7046 hybrid_clock.cc:648] HybridClock initialized: now 1785977531112306 us; error 32 us; skew 500 ppm
Aug 06 00:52:11 dist-test-slave-45dx krb5kdc[5774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977531, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:11.115105 7046 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:52:11.115993 7046 webserver.cc:460] Webserver started at http://127.15.147.131:39439/ using document root <none> and password file <none>
I20260806 00:52:11.116571 7046 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:11.116647 7046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:11.120254 7046 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:52:11.122248 7069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:11.123454 7046 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:52:11.123580 7046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "3453ef5cf4c84d29b927a3eb4fd09b56"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "7280a332ab195ce1e1a8854582f4ee4b"
server_key_iv: "fdb9a9008c9506cc829d14e4ff568d22"
server_key_version: "encryptionkey@0"
I20260806 00:52:11.124122 7046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:11.143679 7046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:11.146425 7046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:11.146631 7046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:11.147249 7046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:11.148089 7046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:11.148176 7046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:11.148241 7046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:11.148275 7046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:11.157732 7046 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:45087
I20260806 00:52:11.157768 7232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:45087 every 8 connection(s)
I20260806 00:52:11.158671 7046 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:52:11.158963 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 7046
I20260806 00:52:11.168030 6847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:11.159708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46214 (local address 127.15.147.190:37145)
0806 00:52:11.159843 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:11.159847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:11.160109 (+ 262us) server_negotiation.cc:407] Connection header received
0806 00:52:11.160245 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:11.160248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:11.160310 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:11.160400 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:11.161320 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.161897 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.162952 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.163284 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.163396 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:11.164856 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:11.164860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:11.164862 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:11.164894 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:11.166379 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:11.166819 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:11.166823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:11.166828 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:11.166904 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:11.167197 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:11.167201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:11.167203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:11.167385 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:11.167603 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:11.167751 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 00:52:11.167861 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 00:52:11 dist-test-slave-45dx krb5kdc[5774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977531, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:52:11.173034 7239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:11.161041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:49901 (local address 127.15.147.190:37145)
0806 00:52:11.161258 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:11.161262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:11.162239 (+ 977us) server_negotiation.cc:407] Connection header received
0806 00:52:11.163837 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:11.163840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:11.163894 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:11.164006 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:11.165304 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.165772 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.166833 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.167059 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.167178 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:11.169974 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:11.169985 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:11.169988 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:11.170011 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:11.171222 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:11.171684 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:11.171686 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:11.171687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:11.171761 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:11.172059 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:11.172061 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:11.172062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:11.172212 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:11.172284 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:11.172786 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 00:52:11.172877 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"spinlock_wait_cycles":1792,"thread_start_us":78,"threads_started":1}
I20260806 00:52:11.173782 7240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:11.161555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:49901)
0806 00:52:11.162096 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:11.162140 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:11.163645 (+ 1505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:11.164187 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:11.164210 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:11.164567 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:11.165156 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:11.165179 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.165901 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.165904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:11.166702 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:11.166711 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.167220 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.167225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:11.167421 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:11.167977 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:52:11.168000 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:11.169805 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:11.171327 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:11.171331 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:11.171341 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:11.171595 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:11.171843 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:11.171845 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:11.171846 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:11.171977 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:11.172343 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:11.172352 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:11.172588 (+ 236us) client_negotiation.cc:769] Sending connection context
0806 00:52:11.172735 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 00:52:11.172961 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":129,"threads_started":1}
I20260806 00:52:11.174889 7234 heartbeater.cc:347] Connected to a master server at 127.15.147.190:37145
I20260806 00:52:11.175079 7234 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:11.175503 7234 heartbeater.cc:511] Master 127.15.147.190:37145 requested a full tablet report, sending...
I20260806 00:52:11.176494 6575 ts_manager.cc:194] Registered new tserver with Master: 3453ef5cf4c84d29b927a3eb4fd09b56 (127.15.147.131:45087)
I20260806 00:52:11.177033 6575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:49901
I20260806 00:52:11.181545 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:52:11.185178 7239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:11.182135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46218 (local address 127.15.147.190:37145)
0806 00:52:11.182244 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:11.182248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:11.182403 (+ 155us) server_negotiation.cc:407] Connection header received
0806 00:52:11.182535 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:11.182537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:11.182621 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:11.182695 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:52:11.182875 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.183346 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.184137 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.184438 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.184520 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:11.184773 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:52:11.184888 (+ 115us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 00:52:11.185087 (+ 199us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46218: token expired
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 00:52:11.185261 7239 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46218: token expired
W20260806 00:52:11.185599 6472 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.147.190:37145)
I20260806 00:52:11.192237 7239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:11.185953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46234 (local address 127.15.147.190:37145)
0806 00:52:11.186066 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:11.186070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:11.186083 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:52:11.186152 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:11.186154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:11.186186 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:11.186246 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:11.186814 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.187255 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.187959 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.188148 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.188217 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:11.189572 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:11.189575 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:11.189577 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:11.189596 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:11.190813 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:11.191210 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:11.191215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:11.191216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:11.191294 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:11.191537 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:11.191539 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:11.191540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:11.191728 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:11.191831 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:11.192037 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 00:52:11.192128 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:52:11.194500 6575 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:52:11.197844 6575 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:52:11.206028 6752 tablet_service.cc:1467] Processing CreateTablet for tablet 1ae05b895c354dd2ad9272a3f97a0172 (DEFAULT_TABLE table=test-table [id=24c1038512004fc79c84828489d9f5f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:11.206697 6752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae05b895c354dd2ad9272a3f97a0172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:11.209340 6696 tablet_bootstrap.cc:492] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb: Bootstrap starting.
I20260806 00:52:11.210179 6696 tablet_bootstrap.cc:654] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:11.210777 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:11.204825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:45087 (local address 127.0.0.1:49550)
0806 00:52:11.204976 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:11.205001 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:11.205089 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:11.206774 (+ 1685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:11.206777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:11.206796 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:11.206890 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:11.206897 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.208205 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.208208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:11.209319 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:11.209334 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.210391 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.210395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:11.210480 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:11.210504 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 00:52:11.210552 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 00:52:11.210644 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":44}
I20260806 00:52:11.211354 7248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:11.205167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:37759 (local address 127.0.0.1:35668)
0806 00:52:11.205673 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:11.205687 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:11.205773 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:11.207034 (+ 1261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:11.207037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:11.207050 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:11.207135 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:11.207141 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.208113 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.208116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:11.209853 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:11.209862 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.210923 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.210927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:11.211024 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:11.211059 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 00:52:11.211130 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 00:52:11.211224 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":31,"thread_start_us":147,"threads_started":1}
I20260806 00:52:11.211447 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:11.205188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49550 (local address 127.15.147.131:45087)
0806 00:52:11.206273 (+ 1085us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:11.206290 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:11.206314 (+ 24us) server_negotiation.cc:407] Connection header received
0806 00:52:11.206393 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:11.206400 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:11.206593 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:11.206867 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:11.207037 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.208100 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.209465 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.210265 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.211054 (+ 789us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:11.211159 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:11.211220 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 00:52:11.211304 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":168,"threads_started":1}
I20260806 00:52:11.211447 7250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:11.205670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35668 (local address 127.15.147.130:37759)
0806 00:52:11.206021 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:11.206033 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:11.206059 (+ 26us) server_negotiation.cc:407] Connection header received
0806 00:52:11.206126 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:11.206131 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:11.206350 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:11.206525 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:11.207282 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.207979 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.209986 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.210804 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.210945 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:11.211096 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:11.211166 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 00:52:11.211284 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":149,"threads_started":1}
I20260806 00:52:11.211436 6696 tablet_bootstrap.cc:492] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb: No bootstrap required, opened a new log
I20260806 00:52:11.211560 6696 ts_tablet_manager.cc:1397] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260806 00:52:11.212265 6696 raft_consensus.cc:348] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:52:11.212422 6696 raft_consensus.cc:374] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:11.212455 6696 raft_consensus.cc:729] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1986982f88094c71b5cbeb60b5e758fb, State: Initialized, Role: FOLLOWER
I20260806 00:52:11.212589 6696 consensus_queue.cc:260] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:52:11.212939 6696 ts_tablet_manager.cc:1428] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 00:52:11.214466 6943 tablet_service.cc:1467] Processing CreateTablet for tablet 1ae05b895c354dd2ad9272a3f97a0172 (DEFAULT_TABLE table=test-table [id=24c1038512004fc79c84828489d9f5f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:11.214941 7140 tablet_service.cc:1467] Processing CreateTablet for tablet 1ae05b895c354dd2ad9272a3f97a0172 (DEFAULT_TABLE table=test-table [id=24c1038512004fc79c84828489d9f5f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:11.215592 6943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae05b895c354dd2ad9272a3f97a0172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:11.215924 7140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae05b895c354dd2ad9272a3f97a0172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:11.223757 7255 tablet_bootstrap.cc:492] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56: Bootstrap starting.
I20260806 00:52:11.224231 7257 tablet_bootstrap.cc:492] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad: Bootstrap starting.
I20260806 00:52:11.226675 7255 tablet_bootstrap.cc:654] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:11.226701 7257 tablet_bootstrap.cc:654] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:11.227584 7255 log.cc:824] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:11.227607 7257 log.cc:824] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:11.229488 7257 tablet_bootstrap.cc:492] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad: No bootstrap required, opened a new log
I20260806 00:52:11.229516 7255 tablet_bootstrap.cc:492] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56: No bootstrap required, opened a new log
I20260806 00:52:11.229691 7257 ts_tablet_manager.cc:1397] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 00:52:11.229715 7255 ts_tablet_manager.cc:1397] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:52:11.232532 7257 raft_consensus.cc:348] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:52:11.232532 7255 raft_consensus.cc:348] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:52:11.232776 7257 raft_consensus.cc:374] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:11.232805 7255 raft_consensus.cc:374] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:11.232833 7257 raft_consensus.cc:729] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64dc15846a89408fab94c25dba93b2ad, State: Initialized, Role: FOLLOWER
I20260806 00:52:11.232878 7255 raft_consensus.cc:729] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3453ef5cf4c84d29b927a3eb4fd09b56, State: Initialized, Role: FOLLOWER
I20260806 00:52:11.233249 7255 consensus_queue.cc:260] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:52:11.233249 7257 consensus_queue.cc:260] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:52:11.233934 7043 heartbeater.cc:503] Master 127.15.147.190:37145 was elected leader, sending a full tablet report...
I20260806 00:52:11.234284 7257 ts_tablet_manager.cc:1428] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260806 00:52:11.234297 7255 ts_tablet_manager.cc:1428] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:52:11.234643 7234 heartbeater.cc:503] Master 127.15.147.190:37145 was elected leader, sending a full tablet report...
W20260806 00:52:11.273443 6845 tablet.cc:2369] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:52:11.409830 7236 tablet.cc:2369] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:52:11.469383 7044 tablet.cc:2369] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:52:11.491360 7262 raft_consensus.cc:482] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:52:11.491570 7262 raft_consensus.cc:504] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:52:11.492924 7262 leader_election.cc:288] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64dc15846a89408fab94c25dba93b2ad (127.15.147.130:37759), 1986982f88094c71b5cbeb60b5e758fb (127.15.147.129:36171)
I20260806 00:52:11.497048 7263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:11.493321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:52675 (local address 127.15.147.129:36171)
0806 00:52:11.493687 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:11.493692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:11.493708 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:52:11.493853 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:11.493856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:11.493913 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:11.493993 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:11.494349 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.494858 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.495979 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.496452 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.496582 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:11.496639 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:11.496846 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 00:52:11.496919 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":95,"threads_started":1}
I20260806 00:52:11.497206 7250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:11.493391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:51293 (local address 127.15.147.130:37759)
0806 00:52:11.493752 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:11.493757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:11.493773 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:52:11.493828 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:11.493831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:11.493889 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:11.494338 (+ 449us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:11.494440 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.494967 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.496051 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.496549 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.496692 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:11.496752 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:11.496977 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 00:52:11.497052 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263}
I20260806 00:52:11.497061 7264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:11.493178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:36171 (local address 127.15.147.131:52675)
0806 00:52:11.493621 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:11.493636 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:11.493737 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:11.494052 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:11.494055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:11.494072 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:11.494222 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:11.494229 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.494989 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.494992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:11.495840 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:11.495849 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.496574 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.496578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:11.496677 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:11.496733 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 00:52:11.496827 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 00:52:11.496904 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":177,"spinlock_wait_cycles":14208,"thread_start_us":116,"threads_started":1}
I20260806 00:52:11.497354 7240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:11.493174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:37759 (local address 127.15.147.131:51293)
0806 00:52:11.493438 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:11.493457 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:11.493565 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:11.494061 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:11.494065 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:11.494086 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:11.494224 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:11.494230 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.495093 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.495099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:11.495900 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:11.495909 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.496683 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.496686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:11.496837 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:11.496855 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 00:52:11.496958 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 00:52:11.497032 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":4}
I20260806 00:52:11.497901 6778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae05b895c354dd2ad9272a3f97a0172" candidate_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1986982f88094c71b5cbeb60b5e758fb" is_pre_election: true
I20260806 00:52:11.497962 6970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae05b895c354dd2ad9272a3f97a0172" candidate_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64dc15846a89408fab94c25dba93b2ad" is_pre_election: true
I20260806 00:52:11.498210 6778 raft_consensus.cc:2454] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3453ef5cf4c84d29b927a3eb4fd09b56 in term 0.
I20260806 00:52:11.498263 6970 raft_consensus.cc:2454] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3453ef5cf4c84d29b927a3eb4fd09b56 in term 0.
I20260806 00:52:11.498677 7072 leader_election.cc:302] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1986982f88094c71b5cbeb60b5e758fb, 3453ef5cf4c84d29b927a3eb4fd09b56; no voters:
I20260806 00:52:11.498974 7262 raft_consensus.cc:2781] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:52:11.499063 7262 raft_consensus.cc:482] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:52:11.499106 7262 raft_consensus.cc:3037] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:11.500272 7262 raft_consensus.cc:504] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:52:11.500607 7262 leader_election.cc:288] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [CANDIDATE]: Term 1 election: Requested vote from peers 64dc15846a89408fab94c25dba93b2ad (127.15.147.130:37759), 1986982f88094c71b5cbeb60b5e758fb (127.15.147.129:36171)
I20260806 00:52:11.500946 6778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae05b895c354dd2ad9272a3f97a0172" candidate_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1986982f88094c71b5cbeb60b5e758fb"
I20260806 00:52:11.500960 6970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae05b895c354dd2ad9272a3f97a0172" candidate_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64dc15846a89408fab94c25dba93b2ad"
I20260806 00:52:11.501106 6778 raft_consensus.cc:3037] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:11.501112 6970 raft_consensus.cc:3037] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:11.502426 6778 raft_consensus.cc:2454] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3453ef5cf4c84d29b927a3eb4fd09b56 in term 1.
I20260806 00:52:11.502451 6970 raft_consensus.cc:2454] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3453ef5cf4c84d29b927a3eb4fd09b56 in term 1.
I20260806 00:52:11.502760 7072 leader_election.cc:302] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1986982f88094c71b5cbeb60b5e758fb, 3453ef5cf4c84d29b927a3eb4fd09b56; no voters:
I20260806 00:52:11.502933 7262 raft_consensus.cc:2781] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:52:11.503141 7262 raft_consensus.cc:686] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 1 LEADER]: Becoming Leader. State: Replica: 3453ef5cf4c84d29b927a3eb4fd09b56, State: Running, Role: LEADER
I20260806 00:52:11.503481 7262 consensus_queue.cc:237] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:52:11.506844 6575 catalog_manager.cc:5495] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3453ef5cf4c84d29b927a3eb4fd09b56 (127.15.147.131). New cstate: current_term: 1 leader_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } health_report { overall_health: UNKNOWN } } }
I20260806 00:52:11.572979 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:11.569194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49552 (local address 127.15.147.131:45087)
0806 00:52:11.569354 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:11.569358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:11.569374 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:52:11.569556 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:11.569559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:11.569660 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:11.569794 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:52:11.570051 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.570554 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.571440 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:11.571669 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:11.571787 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:11.572106 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:52:11.572526 (+ 420us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:52:11.572665 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:11.572728 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 00:52:11.572786 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:52:11.578783 6778 raft_consensus.cc:1261] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Refusing update from remote peer 3453ef5cf4c84d29b927a3eb4fd09b56: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:52:11.578788 6970 raft_consensus.cc:1261] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad [term 1 FOLLOWER]: Refusing update from remote peer 3453ef5cf4c84d29b927a3eb4fd09b56: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:52:11.579537 7266 consensus_queue.cc:1035] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:52:11.579738 7262 consensus_queue.cc:1035] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:52:11.585697 7270 mvcc.cc:205] Tried to move back new op lower bound from 7315363969339269120 to 7315363969040793600. Current Snapshot: MvccSnapshot[applied={T|T < 7315363969339269120}]
I20260806 00:52:11.587313 7273 mvcc.cc:205] Tried to move back new op lower bound from 7315363969339269120 to 7315363969040793600. Current Snapshot: MvccSnapshot[applied={T|T < 7315363969339269120}]
I20260806 00:52:11.595172 6574 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:52:11.595314 6574 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:52:11.597606 6574 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: Sending DeleteTablet for 3 replicas of tablet 1ae05b895c354dd2ad9272a3f97a0172
I20260806 00:52:11.598281 7140 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ae05b895c354dd2ad9272a3f97a0172 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:11 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:49550
I20260806 00:52:11.598326 6943 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ae05b895c354dd2ad9272a3f97a0172 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:11 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:35668
I20260806 00:52:11.598433 6752 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ae05b895c354dd2ad9272a3f97a0172 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:11 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:44202
I20260806 00:52:11.598838 7289 tablet_replica.cc:330] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb: stopping tablet replica
I20260806 00:52:11.598974 7289 raft_consensus.cc:2229] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:52:11.598984 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 6630
I20260806 00:52:11.599124 7289 raft_consensus.cc:2258] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:11.599406 7286 tablet_replica.cc:330] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56: stopping tablet replica
I20260806 00:52:11.599853 7286 raft_consensus.cc:2229] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:52:11.600358 7286 raft_consensus.cc:2258] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:11.600802 7288 tablet_replica.cc:330] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad: stopping tablet replica
I20260806 00:52:11.601163 7288 raft_consensus.cc:2229] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:52:11.601576 7288 raft_consensus.cc:2258] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:11.604018 7286 ts_tablet_manager.cc:1905] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:52:11.608256 7288 ts_tablet_manager.cc:1905] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:52:11.610401 7288 ts_tablet_manager.cc:1918] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:52:11.610525 7288 log.cc:1196] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/1ae05b895c354dd2ad9272a3f97a0172
I20260806 00:52:11.610810 7288 ts_tablet_manager.cc:1939] T 1ae05b895c354dd2ad9272a3f97a0172 P 64dc15846a89408fab94c25dba93b2ad: Deleting consensus metadata
I20260806 00:52:11.612130 7286 ts_tablet_manager.cc:1918] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:52:11.612155 6554 catalog_manager.cc:4841] TS 64dc15846a89408fab94c25dba93b2ad (127.15.147.130:37759): tablet 1ae05b895c354dd2ad9272a3f97a0172 (table test-table [id=24c1038512004fc79c84828489d9f5f3]) successfully deleted
I20260806 00:52:11.612238 7286 log.cc:1196] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/1ae05b895c354dd2ad9272a3f97a0172
I20260806 00:52:11.612777 7286 ts_tablet_manager.cc:1939] T 1ae05b895c354dd2ad9272a3f97a0172 P 3453ef5cf4c84d29b927a3eb4fd09b56: Deleting consensus metadata
I20260806 00:52:11.613830 6554 catalog_manager.cc:4841] TS 3453ef5cf4c84d29b927a3eb4fd09b56 (127.15.147.131:45087): tablet 1ae05b895c354dd2ad9272a3f97a0172 (table test-table [id=24c1038512004fc79c84828489d9f5f3]) successfully deleted
W20260806 00:52:11.614517 6555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.147.129:36171 (error 108)
I20260806 00:52:11.615114 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 6850
I20260806 00:52:11.617344 7248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:11.615313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:36171 (local address 127.0.0.1:44206)
0806 00:52:11.617153 (+ 1840us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:11.617238 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:36171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1734}
W20260806 00:52:11.617559 6555 catalog_manager.cc:4584] TS 1986982f88094c71b5cbeb60b5e758fb (127.15.147.129:36171): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ae05b895c354dd2ad9272a3f97a0172: Network error: Client connection negotiation failed: client connection to 127.15.147.129:36171: connect: Connection refused (error 111)
I20260806 00:52:11.626067 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 7046
I20260806 00:52:11.636677 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 6534
I20260806 00:52:31.649936 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:37145
--webserver_interface=127.15.147.190
--webserver_port=40589
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:37145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:52:31.765733 7291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:31.765971 7291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:31.766013 7291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:31.770063 7291 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 00:52:31.770110 7291 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 00:52:31.770126 7291 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 00:52:31.770140 7291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:52:31.770157 7291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:31.770211 7291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:31.770239 7291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:52:31.770270 7291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:52:31.775457 7291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:37145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:37145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=40589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:31.776574 7291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:31.777690 7291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:31.786809 7297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:31.786871 7298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:31.786890 7291 server_base.cc:1034] running on GCE node
W20260806 00:52:31.787065 7301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:31.787699 7291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:31.788853 7291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:31.790017 7291 hybrid_clock.cc:648] HybridClock initialized: now 1785977551789997 us; error 32 us; skew 500 ppm
Aug 06 00:52:31 dist-test-slave-45dx krb5kdc[5774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977551, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:31.800562 7291 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:52:31.801515 7291 webserver.cc:460] Webserver started at http://127.15.147.190:40589/ using document root <none> and password file <none>
I20260806 00:52:31.802094 7291 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:31.802158 7291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:31.805963 7291 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260806 00:52:31.808146 7312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:31.809434 7291 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:52:31.809592 7291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "52d02cd310a248c6a4a40d2a82c7fe39"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "95da7076172494966d00e75f44d3454e"
server_key_iv: "25b289aa9d92aae3fa28147c75f41bac"
server_key_version: "encryptionkey@0"
I20260806 00:52:31.810151 7291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:31.831844 7291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:31.834797 7291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:31.835016 7291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:31.842453 7291 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:37145
I20260806 00:52:31.842751 7387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:37145 every 8 connection(s)
I20260806 00:52:31.843546 7291 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:52:31.847775 7388 sys_catalog.cc:263] Verifying existing consensus state
I20260806 00:52:31.848412 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 7291
I20260806 00:52:31.848839 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:36171
--local_ip_for_outbound_sockets=127.15.147.129
--tserver_master_addrs=127.15.147.190:37145
--webserver_port=35307
--webserver_interface=127.15.147.129
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:52:31.849611 7388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: Bootstrap starting.
I20260806 00:52:31.868132 7388 log.cc:824] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:31.879441 7388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:52:31.879843 7388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: Bootstrap complete.
I20260806 00:52:31.883996 7388 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37145 } }
I20260806 00:52:31.884479 7388 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52d02cd310a248c6a4a40d2a82c7fe39, State: Initialized, Role: FOLLOWER
I20260806 00:52:31.885010 7388 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37145 } }
I20260806 00:52:31.885155 7388 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:52:31.885241 7388 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:52:31.885357 7388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 2 FOLLOWER]: Advancing to term 3
I20260806 00:52:31.887226 7388 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37145 } }
I20260806 00:52:31.887566 7388 leader_election.cc:302] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52d02cd310a248c6a4a40d2a82c7fe39; no voters:
I20260806 00:52:31.887908 7388 leader_election.cc:288] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 00:52:31.888025 7393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 00:52:31.888244 7393 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [term 3 LEADER]: Becoming Leader. State: Replica: 52d02cd310a248c6a4a40d2a82c7fe39, State: Running, Role: LEADER
I20260806 00:52:31.888622 7393 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37145 } }
I20260806 00:52:31.889182 7388 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:52:31.889544 7393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52d02cd310a248c6a4a40d2a82c7fe39. Latest consensus state: current_term: 3 leader_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37145 } } }
I20260806 00:52:31.889669 7393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [sys.catalog]: This master's current role is: LEADER
I20260806 00:52:31.890369 7394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52d02cd310a248c6a4a40d2a82c7fe39" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 37145 } } }
I20260806 00:52:31.890475 7394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39 [sys.catalog]: This master's current role is: LEADER
I20260806 00:52:31.890981 7397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:52:31.896108 7397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:52:31.896867 7397 catalog_manager.cc:1259] Loaded cluster ID: ea9569e939e44008b4967a519dbe8d96
I20260806 00:52:31.896934 7397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:52:31.899529 7397 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:52:31.900321 7397 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: Loaded TSK: 0
I20260806 00:52:31.900477 7397 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 52d02cd310a248c6a4a40d2a82c7fe39: Loaded TSK: 1
F20260806 00:52:31.901333 7397 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fae13bbf9ad google::LogMessage::Fail() at ??:0
@ 0x7fae13bc288a google::LogMessage::SendToLog() at ??:0
@ 0x7fae13bbf4de google::LogMessage::Flush() at ??:0
@ 0x7fae13bc0a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fae1688318d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fae1687bf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fae1e5247bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fae1e4fe99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fae1e4ff178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fae1e540df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fae1e55f82b std::function<>::operator()() at ??:0
@ 0x7fae1e4feb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fae1e4fff3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fae1e4fc579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fae1e540640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fae15e113d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fae15e11c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fae15e133f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fae15e02512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fae174466db start_thread at ??:0
@ 0x7fae126bd61f clone at ??:0
W20260806 00:52:32.001379 7390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:32.001812 7390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:32.001936 7390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:32.010223 7390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:32.010294 7390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:32.010418 7390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:52:32.016209 7390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:36171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=35307
--tserver_master_addrs=127.15.147.190:37145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:32.017441 7390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:32.018548 7390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:52:32.028733 7390 server_base.cc:1034] running on GCE node
W20260806 00:52:32.028694 7454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:32.028693 7457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:32.029057 7460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:32.029503 7390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:32.030194 7390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:32.031381 7390 hybrid_clock.cc:648] HybridClock initialized: now 1785977552031347 us; error 43 us; skew 500 ppm
Aug 06 00:52:32 dist-test-slave-45dx krb5kdc[5774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977552, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:32.034315 7390 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:52:32.035308 7390 webserver.cc:460] Webserver started at http://127.15.147.129:35307/ using document root <none> and password file <none>
I20260806 00:52:32.035878 7390 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:32.035976 7390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:32.039923 7390 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:52:32.042214 7470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:32.043486 7390 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:52:32.043583 7390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "1986982f88094c71b5cbeb60b5e758fb"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "7f4fb4284e9769de99101ea717946bce"
server_key_iv: "bb87c240e8c04053f8937a76fab47495"
server_key_version: "encryptionkey@0"
I20260806 00:52:32.044142 7390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:32.063269 7390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:32.066277 7390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:32.066493 7390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:32.067143 7390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:32.068224 7481 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 00:52:32.072191 7390 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 00:52:32.072252 7390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260806 00:52:32.072340 7390 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 00:52:32.073997 7390 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 00:52:32.074055 7390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 00:52:32.074331 7481 tablet_bootstrap.cc:492] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb: Bootstrap starting.
I20260806 00:52:32.085563 7390 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:36171
I20260806 00:52:32.085809 7646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:36171 every 8 connection(s)
I20260806 00:52:32.086753 7390 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:52:32.087320 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 7390
I20260806 00:52:32.087815 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:37759
--local_ip_for_outbound_sockets=127.15.147.130
--tserver_master_addrs=127.15.147.190:37145
--webserver_port=36833
--webserver_interface=127.15.147.130
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:52:32.097491 7481 log.cc:824] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:32.106856 7481 tablet_bootstrap.cc:492] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 00:52:32.107275 7481 tablet_bootstrap.cc:492] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb: Bootstrap complete.
I20260806 00:52:32.108944 7481 ts_tablet_manager.cc:1397] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb: Time spent bootstrapping tablet: real 0.035s user 0.026s sys 0.008s
I20260806 00:52:32.112383 7481 raft_consensus.cc:348] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:52:32.112916 7481 raft_consensus.cc:729] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1986982f88094c71b5cbeb60b5e758fb, State: Initialized, Role: FOLLOWER
I20260806 00:52:32.113430 7481 consensus_queue.cc:260] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:52:32.114500 7481 ts_tablet_manager.cc:1428] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260806 00:52:32.134229 7651 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:52:32.089733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:42227)
0806 00:52:32.092144 (+ 2411us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:32.092203 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:32.093314 (+ 1111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:32.133252 (+ 39938us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":2166,"thread_start_us":183,"threads_started":1}
W20260806 00:52:32.134843 7648 heartbeater.cc:644] Failed to heartbeat to 127.15.147.190:37145 (0 consecutive failures): Network error: Failed to ping master at 127.15.147.190:37145: Client connection negotiation failed: client connection to 127.15.147.190:37145: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 00:52:32.135679 7651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:32.135390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:37385)
0806 00:52:32.135533 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:32.135609 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 00:52:32.136432 7651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:32.136180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:47233)
0806 00:52:32.136321 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:32.136376 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 00:52:32.136698 7648 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 00:52:32.211766 7650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:32.212142 7650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:32.212230 7650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:32.216173 7650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:32.216259 7650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:32.216363 7650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:52:32.221405 7650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:37759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=36833
--tserver_master_addrs=127.15.147.190:37145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:32.222414 7650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:32.223435 7650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:32.233264 7662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:32.233243 7666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:32.233223 7664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:32.233646 7650 server_base.cc:1034] running on GCE node
I20260806 00:52:32.234071 7650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:32.234642 7650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:32.235819 7650 hybrid_clock.cc:648] HybridClock initialized: now 1785977552235804 us; error 27 us; skew 500 ppm
Aug 06 00:52:32 dist-test-slave-45dx krb5kdc[5774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977552, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:32.242308 7650 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:52:32.243201 7650 webserver.cc:460] Webserver started at http://127.15.147.130:36833/ using document root <none> and password file <none>
I20260806 00:52:32.243762 7650 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:32.243862 7650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:32.247670 7650 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:52:32.249905 7675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:32.251273 7650 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:52:32.251415 7650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "64dc15846a89408fab94c25dba93b2ad"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "f5992550baaa1be418eccb03866b4967"
server_key_iv: "4c2fcaed1d9918726b943feb2cb110ff"
server_key_version: "encryptionkey@0"
I20260806 00:52:32.251982 7650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:32.290730 7650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:32.295102 7650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:32.295383 7650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:32.296351 7650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:32.297659 7650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:32.297739 7650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:32.297816 7650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:32.297859 7650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:32.312566 7650 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:37759
I20260806 00:52:32.312642 7849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:37759 every 8 connection(s)
I20260806 00:52:32.314226 7650 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:52:32.319098 7854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:32.317205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:57385)
0806 00:52:32.317840 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:32.317948 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":197,"threads_started":1}
W20260806 00:52:32.319643 7851 heartbeater.cc:644] Failed to heartbeat to 127.15.147.190:37145 (0 consecutive failures): Network error: Failed to ping master at 127.15.147.190:37145: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
I20260806 00:52:32.320339 7854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:32.320045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:42629)
0806 00:52:32.320178 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:32.320244 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 00:52:32.321025 7854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:32.320765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:45753)
0806 00:52:32.320888 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:32.320944 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 00:52:32.321230 7851 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 00:52:32.322115 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 7650
I20260806 00:52:32.322567 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:45087
--local_ip_for_outbound_sockets=127.15.147.131
--tserver_master_addrs=127.15.147.190:37145
--webserver_port=39439
--webserver_interface=127.15.147.131
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:52:32.437359 7855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:32.437613 7855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:32.437678 7855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:32.441555 7855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:32.441639 7855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:32.441717 7855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:52:32.446795 7855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:45087
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=39439
--tserver_master_addrs=127.15.147.190:37145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:32.447726 7855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:32.448700 7855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:32.458869 7865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:32.458957 7864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:32.458891 7867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:32.459303 7855 server_base.cc:1034] running on GCE node
I20260806 00:52:32.459683 7855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:32.460284 7855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:32.461505 7855 hybrid_clock.cc:648] HybridClock initialized: now 1785977552461484 us; error 33 us; skew 500 ppm
Aug 06 00:52:32 dist-test-slave-45dx krb5kdc[5774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977552, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:32.464247 7855 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:52:32.465102 7855 webserver.cc:460] Webserver started at http://127.15.147.131:39439/ using document root <none> and password file <none>
I20260806 00:52:32.465639 7855 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:32.465703 7855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:32.469393 7855 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 00:52:32.471400 7876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:32.472721 7855 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:52:32.472872 7855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "3453ef5cf4c84d29b927a3eb4fd09b56"
format_stamp: "Formatted at 2026-08-06 00:51:49 on dist-test-slave-45dx"
server_key: "7280a332ab195ce1e1a8854582f4ee4b"
server_key_iv: "fdb9a9008c9506cc829d14e4ff568d22"
server_key_version: "encryptionkey@0"
I20260806 00:52:32.473424 7855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:32.493563 7855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:32.497076 7855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:32.497295 7855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:32.497915 7855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:32.498759 7855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:32.498833 7855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:32.498919 7855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:32.498970 7855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:32.508008 7855 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:45087
I20260806 00:52:32.508085 8039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:45087 every 8 connection(s)
I20260806 00:52:32.509095 7855 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:52:32.511659 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 7855
I20260806 00:52:32.513528 8043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:32.511238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:47225)
0806 00:52:32.511833 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:32.511980 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":150,"threads_started":1}
W20260806 00:52:32.514001 8041 heartbeater.cc:644] Failed to heartbeat to 127.15.147.190:37145 (0 consecutive failures): Network error: Failed to ping master at 127.15.147.190:37145: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
I20260806 00:52:32.514562 8043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:32.514343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:45883)
0806 00:52:32.514447 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:32.514506 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 00:52:32.515277 8043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:32.515046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:53949)
0806 00:52:32.515168 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:32.515223 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 00:52:32.515547 8041 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 00:52:33.137696 8045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:33.137282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:38591)
0806 00:52:33.137528 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:33.137583 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":77,"threads_started":1}
I20260806 00:52:33.322577 8047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:33.321936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:39335)
0806 00:52:33.322400 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:33.322461 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":238,"threads_started":1}
I20260806 00:52:33.516645 8048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:33.516197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:39745)
0806 00:52:33.516481 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:33.516542 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":118,"threads_started":1}
I20260806 00:52:33.530237 8049 raft_consensus.cc:482] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:52:33.530462 8049 raft_consensus.cc:504] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:52:33.531479 8049 leader_election.cc:288] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3453ef5cf4c84d29b927a3eb4fd09b56 (127.15.147.131:45087), 64dc15846a89408fab94c25dba93b2ad (127.15.147.130:37759)
Aug 06 00:52:33 dist-test-slave-45dx krb5kdc[5774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977552, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.130@KRBTEST.COM
Aug 06 00:52:33 dist-test-slave-45dx krb5kdc[5774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977552, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.131@KRBTEST.COM
I20260806 00:52:33.548113 8052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:33.531988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:37759 (local address 127.15.147.129:49625)
0806 00:52:33.532365 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:33.532380 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:33.532486 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:33.534206 (+ 1720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:33.534212 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:33.534871 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:33.535737 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:33.535759 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:33.537699 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:33.537703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:33.539299 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:33.539312 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:33.540283 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:33.540287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:33.540631 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:33.541441 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:52:33.541467 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:33.543437 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:33.545633 (+ 2196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:33.545638 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:33.545651 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:33.546005 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:33.546357 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:33.546360 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:33.546362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:33.546506 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:33.547142 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:33.547148 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:33.547416 (+ 268us) client_negotiation.cc:769] Sending connection context
0806 00:52:33.547599 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 00:52:33.547904 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":408,"spinlock_wait_cycles":5248,"thread_start_us":72,"threads_started":1}
I20260806 00:52:33.548161 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:33.532175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:49625 (local address 127.15.147.130:37759)
0806 00:52:33.532556 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:33.532581 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:33.532616 (+ 35us) server_negotiation.cc:407] Connection header received
0806 00:52:33.533407 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:33.533414 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:33.533768 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:33.534104 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:33.535948 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:33.537535 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:33.539443 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:33.540129 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:33.540299 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:33.543709 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:33.543714 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:33.543726 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:33.543753 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:33.545512 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:33.546126 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:33.546130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:33.546134 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:33.546244 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:33.546603 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:33.546608 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:33.546610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:33.547005 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:33.547203 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:33.547724 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 00:52:33.547997 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":120,"threads_started":1}
I20260806 00:52:33.548445 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:33.531869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:33155 (local address 127.15.147.131:45087)
0806 00:52:33.532236 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:33.532267 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:33.532378 (+ 111us) server_negotiation.cc:407] Connection header received
0806 00:52:33.533315 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:33.533327 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:33.533769 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:33.534104 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:33.535948 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:33.537536 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:33.539443 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:33.540129 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:33.540320 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:33.543977 (+ 3657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:33.543983 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:33.543993 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:33.544021 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:33.546073 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:33.546475 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:33.546488 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:33.546497 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:33.546607 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:33.546893 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:33.546897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:33.546898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:33.547406 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:33.547553 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:33.547919 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 00:52:33.548198 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":84,"threads_started":1}
I20260806 00:52:33.548184 8045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:33.531688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:45087 (local address 127.15.147.129:33155)
0806 00:52:33.531824 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:33.531840 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:33.531954 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:33.534206 (+ 2252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:33.534212 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:33.534871 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:33.535737 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:33.535759 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:33.537699 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:33.537703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:33.539299 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:33.539312 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:33.540283 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:33.540287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:33.540631 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:33.541435 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:52:33.541468 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:33.543783 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:33.546166 (+ 2383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:33.546170 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:33.546172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:33.546362 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:33.546711 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:33.546714 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:33.546716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:33.546802 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:33.547538 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:33.547541 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:33.547646 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 00:52:33.547753 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 00:52:33.548008 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":492,"spinlock_wait_cycles":42240}
W20260806 00:52:33.551152 7474 leader_election.cc:341] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3453ef5cf4c84d29b927a3eb4fd09b56 (127.15.147.131:45087): Not found: Tablet not found: 1ae05b895c354dd2ad9272a3f97a0172
W20260806 00:52:33.551410 7474 leader_election.cc:341] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64dc15846a89408fab94c25dba93b2ad (127.15.147.130:37759): Not found: Tablet not found: 1ae05b895c354dd2ad9272a3f97a0172
I20260806 00:52:33.551465 7474 leader_election.cc:302] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1986982f88094c71b5cbeb60b5e758fb; no voters: 3453ef5cf4c84d29b927a3eb4fd09b56, 64dc15846a89408fab94c25dba93b2ad
I20260806 00:52:33.551721 8049 raft_consensus.cc:2726] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:52:34.138911 8057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:34.138491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:37695)
0806 00:52:34.138751 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:34.138811 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":75,"threads_started":1}
I20260806 00:52:34.323945 8058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:34.323522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:39739)
0806 00:52:34.323776 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:34.323825 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":95,"threads_started":1}
I20260806 00:52:34.517736 8059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:34.517359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:47631)
0806 00:52:34.517596 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:34.517650 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":74,"threads_started":1}
I20260806 00:52:35.140429 8061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:35.139823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:39737)
0806 00:52:35.140149 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:35.140235 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":97,"threads_started":1}
I20260806 00:52:35.326478 8063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:35.324732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:35143)
0806 00:52:35.326331 (+ 1599us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:35.326385 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1522,"thread_start_us":1314,"threads_started":1}
I20260806 00:52:35.518901 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:35.518435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:35807)
0806 00:52:35.518725 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:35.518784 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":93,"threads_started":1}
I20260806 00:52:35.756157 8065 raft_consensus.cc:482] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:52:35.756354 8065 raft_consensus.cc:504] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:52:35.756740 8065 leader_election.cc:288] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3453ef5cf4c84d29b927a3eb4fd09b56 (127.15.147.131:45087), 64dc15846a89408fab94c25dba93b2ad (127.15.147.130:37759)
W20260806 00:52:35.757663 7474 leader_election.cc:341] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3453ef5cf4c84d29b927a3eb4fd09b56 (127.15.147.131:45087): Not found: Tablet not found: 1ae05b895c354dd2ad9272a3f97a0172
W20260806 00:52:35.757844 7474 leader_election.cc:341] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64dc15846a89408fab94c25dba93b2ad (127.15.147.130:37759): Not found: Tablet not found: 1ae05b895c354dd2ad9272a3f97a0172
I20260806 00:52:35.757907 7474 leader_election.cc:302] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1986982f88094c71b5cbeb60b5e758fb; no voters: 3453ef5cf4c84d29b927a3eb4fd09b56, 64dc15846a89408fab94c25dba93b2ad
I20260806 00:52:35.758091 8065 raft_consensus.cc:2726] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:52:36.141590 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:36.141168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:41819)
0806 00:52:36.141438 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:36.141492 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":77,"threads_started":1}
I20260806 00:52:36.327672 8069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:36.327257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:38659)
0806 00:52:36.327522 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:36.327576 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":79,"threads_started":1}
I20260806 00:52:36.520180 8070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:36.519732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:38747)
0806 00:52:36.520032 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:36.520093 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":106,"threads_started":1}
I20260806 00:52:37.142967 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:37.142488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:34977)
0806 00:52:37.142799 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:37.142856 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":77,"threads_started":1}
I20260806 00:52:37.330633 8072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:37.328606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:33011)
0806 00:52:37.330431 (+ 1825us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:37.330492 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1745,"thread_start_us":1576,"threads_started":1}
I20260806 00:52:37.521443 8074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:37.520997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:33719)
0806 00:52:37.521260 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:37.521346 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":92,"threads_started":1}
I20260806 00:52:38.144204 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:38.143806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:45105)
0806 00:52:38.144056 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:38.144108 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":77,"threads_started":1}
I20260806 00:52:38.332065 8077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:38.331583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:44991)
0806 00:52:38.331892 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:38.331961 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":75,"threads_started":1}
I20260806 00:52:38.471832 8079 raft_consensus.cc:482] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:52:38.471975 8079 raft_consensus.cc:504] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:52:38.472460 8079 leader_election.cc:288] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3453ef5cf4c84d29b927a3eb4fd09b56 (127.15.147.131:45087), 64dc15846a89408fab94c25dba93b2ad (127.15.147.130:37759)
W20260806 00:52:38.473326 7474 leader_election.cc:341] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3453ef5cf4c84d29b927a3eb4fd09b56 (127.15.147.131:45087): Not found: Tablet not found: 1ae05b895c354dd2ad9272a3f97a0172
W20260806 00:52:38.473515 7474 leader_election.cc:341] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64dc15846a89408fab94c25dba93b2ad (127.15.147.130:37759): Not found: Tablet not found: 1ae05b895c354dd2ad9272a3f97a0172
I20260806 00:52:38.473592 7474 leader_election.cc:302] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1986982f88094c71b5cbeb60b5e758fb; no voters: 3453ef5cf4c84d29b927a3eb4fd09b56, 64dc15846a89408fab94c25dba93b2ad
I20260806 00:52:38.473791 8079 raft_consensus.cc:2726] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:52:38.522616 8082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:38.522201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:43901)
0806 00:52:38.522464 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:38.522521 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":95,"threads_started":1}
I20260806 00:52:39.145457 8083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:39.145033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:54111)
0806 00:52:39.145304 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:39.145359 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":97,"threads_started":1}
I20260806 00:52:39.333403 8084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:39.332910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:38385)
0806 00:52:39.333223 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:39.333280 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":76,"threads_started":1}
I20260806 00:52:39.523806 8086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:39.523314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:36487)
0806 00:52:39.523649 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:39.523707 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":75,"threads_started":1}
I20260806 00:52:40.146734 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:40.146214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:45597)
0806 00:52:40.146501 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:40.146560 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":83,"threads_started":1}
I20260806 00:52:40.334650 8088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:40.334156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:46373)
0806 00:52:40.334501 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:40.334558 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":91,"threads_started":1}
I20260806 00:52:40.525029 8089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:40.524575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:46867)
0806 00:52:40.524872 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:40.524930 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":99,"threads_started":1}
I20260806 00:52:41.147961 8090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:41.147564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:33513)
0806 00:52:41.147807 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:41.147862 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":76,"threads_started":1}
I20260806 00:52:41.335973 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:41.335538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:51197)
0806 00:52:41.335811 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:41.335871 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":87,"threads_started":1}
I20260806 00:52:41.526345 8092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:41.525915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:35329)
0806 00:52:41.526185 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:41.526238 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":89,"threads_started":1}
I20260806 00:52:42.149425 8093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:42.148890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:35473)
0806 00:52:42.149199 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:42.149284 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":93,"threads_started":1}
I20260806 00:52:42.338198 8096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:42.336994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:40793)
0806 00:52:42.338031 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:42.338100 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":957,"thread_start_us":871,"threads_started":1}
I20260806 00:52:42.527940 8098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:42.527356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:34909)
0806 00:52:42.527708 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:42.527772 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":104,"threads_started":1}
I20260806 00:52:43.075554 8099 raft_consensus.cc:482] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:52:43.075701 8099 raft_consensus.cc:504] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3453ef5cf4c84d29b927a3eb4fd09b56" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 45087 } } peers { permanent_uuid: "64dc15846a89408fab94c25dba93b2ad" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 37759 } } peers { permanent_uuid: "1986982f88094c71b5cbeb60b5e758fb" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 36171 } }
I20260806 00:52:43.076154 8099 leader_election.cc:288] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3453ef5cf4c84d29b927a3eb4fd09b56 (127.15.147.131:45087), 64dc15846a89408fab94c25dba93b2ad (127.15.147.130:37759)
W20260806 00:52:43.077011 7474 leader_election.cc:341] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3453ef5cf4c84d29b927a3eb4fd09b56 (127.15.147.131:45087): Not found: Tablet not found: 1ae05b895c354dd2ad9272a3f97a0172
W20260806 00:52:43.077159 7474 leader_election.cc:341] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 64dc15846a89408fab94c25dba93b2ad (127.15.147.130:37759): Not found: Tablet not found: 1ae05b895c354dd2ad9272a3f97a0172
I20260806 00:52:43.077217 7474 leader_election.cc:302] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1986982f88094c71b5cbeb60b5e758fb; no voters: 3453ef5cf4c84d29b927a3eb4fd09b56, 64dc15846a89408fab94c25dba93b2ad
I20260806 00:52:43.077354 8099 raft_consensus.cc:2726] T 1ae05b895c354dd2ad9272a3f97a0172 P 1986982f88094c71b5cbeb60b5e758fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 00:52:43.150763 8100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:43.150290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:53881)
0806 00:52:43.150598 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:43.150660 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":81,"threads_started":1}
I20260806 00:52:43.339447 8101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:43.339029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:38053)
0806 00:52:43.339294 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:43.339348 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":97,"threads_started":1}
I20260806 00:52:43.529153 8102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:43.528725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:36771)
0806 00:52:43.529006 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:43.529061 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":93,"threads_started":1}
I20260806 00:52:44.151917 8103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:44.151492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:55567)
0806 00:52:44.151748 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:44.151801 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":94,"threads_started":1}
I20260806 00:52:44.340813 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:44.340355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:58593)
0806 00:52:44.340651 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:44.340712 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":73,"threads_started":1}
I20260806 00:52:44.530313 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:44.529888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:36299)
0806 00:52:44.530161 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:44.530216 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":92,"threads_started":1}
I20260806 00:52:45.153220 8109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:45.152728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:45581)
0806 00:52:45.153043 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:45.153105 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":93,"threads_started":1}
I20260806 00:52:45.342021 8111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:45.341593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:34415)
0806 00:52:45.341867 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:45.341922 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":91,"threads_started":1}
I20260806 00:52:45.531498 8112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:45.531088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:42629)
0806 00:52:45.531336 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:45.531392 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":78,"threads_started":1}
I20260806 00:52:46.154421 8113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:46.153974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:57475)
0806 00:52:46.154244 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:46.154301 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":82,"threads_started":1}
I20260806 00:52:46.343531 8114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:46.343020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:59931)
0806 00:52:46.343354 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:46.343421 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":131,"threads_started":1}
I20260806 00:52:46.532666 8116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:46.532174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:55523)
0806 00:52:46.532511 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:46.532568 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":92,"threads_started":1}
I20260806 00:52:47.155817 8117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:47.155396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.129:33067)
0806 00:52:47.155670 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:47.155722 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":100,"threads_started":1}
I20260806 00:52:47.345036 8118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:47.344604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.130:56947)
0806 00:52:47.344863 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:47.344918 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":82,"threads_started":1}
I20260806 00:52:47.520009 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 7390
I20260806 00:52:47.534098 8119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:47.533656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:37145 (local address 127.15.147.131:50155)
0806 00:52:47.533927 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:47.533999 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.190:37145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":93,"threads_started":1}
I20260806 00:52:47.536013 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 7650
I20260806 00:52:47.545385 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 7855
2026-08-06T00:52:47Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (60020 ms)
[----------] 30 tests from SecurityITest (250506 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:52:47 dist-test-slave-45dx krb5kdc[8125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:52:47 dist-test-slave-45dx krb5kdc[8125](info): set up 2 sockets
Aug 06 00:52:47 dist-test-slave-45dx krb5kdc[8125](info): commencing operation
krb5kdc: starting...
W20260806 00:52:48.748387 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:52:48 dist-test-slave-45dx krb5kdc[8125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:52:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:52:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:52:48.881127 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:34673
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:40269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:34673
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:52:48.995883 8143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:48.996225 8143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:48.996327 8143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:49.000705 8143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:52:49.000793 8143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:49.000833 8143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:49.000869 8143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:52:49.000908 8143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:52:49.006245 8143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:34673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:34673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:49.007369 8143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:49.008546 8143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:49.017179 8155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:49.017242 8149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:49.017171 8152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:49.017377 8143 server_base.cc:1034] running on GCE node
I20260806 00:52:49.018085 8143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:49.019166 8143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:49.020364 8143 hybrid_clock.cc:648] HybridClock initialized: now 1785977569020342 us; error 32 us; skew 500 ppm
Aug 06 00:52:49 dist-test-slave-45dx krb5kdc[8125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977569, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:49.023259 8143 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:52:49.024187 8143 webserver.cc:460] Webserver started at http://127.15.147.190:40069/ using document root <none> and password file <none>
I20260806 00:52:49.024802 8143 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:49.024905 8143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:49.025142 8143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:49.027168 8143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "57123e94c61548f6860746951767d8fd"
format_stamp: "Formatted at 2026-08-06 00:52:49 on dist-test-slave-45dx"
server_key: "3bcaa33e466f3f8b407e0fcf0d73aad8"
server_key_iv: "1c563f6bc2e8f4db8f549837fdb218b7"
server_key_version: "encryptionkey@0"
I20260806 00:52:49.027763 8143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "57123e94c61548f6860746951767d8fd"
format_stamp: "Formatted at 2026-08-06 00:52:49 on dist-test-slave-45dx"
server_key: "3bcaa33e466f3f8b407e0fcf0d73aad8"
server_key_iv: "1c563f6bc2e8f4db8f549837fdb218b7"
server_key_version: "encryptionkey@0"
I20260806 00:52:49.031600 8143 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:52:49.034020 8167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:49.035275 8143 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 00:52:49.035419 8143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "57123e94c61548f6860746951767d8fd"
format_stamp: "Formatted at 2026-08-06 00:52:49 on dist-test-slave-45dx"
server_key: "3bcaa33e466f3f8b407e0fcf0d73aad8"
server_key_iv: "1c563f6bc2e8f4db8f549837fdb218b7"
server_key_version: "encryptionkey@0"
I20260806 00:52:49.035533 8143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:49.051103 8143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:49.054091 8143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:49.054296 8143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:49.061717 8143 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:34673
I20260806 00:52:49.061741 8232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:34673 every 8 connection(s)
I20260806 00:52:49.062666 8143 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:52:49.065448 8233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:49.068768 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 8143
I20260806 00:52:49.068920 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:52:49.069273 15950 external_mini_cluster.cc:1428] Setting key 11e089146c4515a16a5425e5275980f2
I20260806 00:52:49.071878 8233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd: Bootstrap starting.
I20260806 00:52:49.074119 8233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:49.074908 8233 log.cc:824] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:49.076882 8233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd: No bootstrap required, opened a new log
Aug 06 00:52:49 dist-test-slave-45dx krb5kdc[8125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:52:49.081292 8233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57123e94c61548f6860746951767d8fd" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34673 } }
I20260806 00:52:49.081617 8233 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:49.081691 8233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57123e94c61548f6860746951767d8fd, State: Initialized, Role: FOLLOWER
I20260806 00:52:49.082151 8233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57123e94c61548f6860746951767d8fd" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34673 } }
I20260806 00:52:49.082275 8233 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:52:49.082368 8233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:52:49.082545 8233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:49.083563 8233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57123e94c61548f6860746951767d8fd" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34673 } }
I20260806 00:52:49.083909 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:49.070716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44674 (local address 127.15.147.190:34673)
0806 00:52:49.071164 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:49.071179 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:49.071215 (+ 36us) server_negotiation.cc:407] Connection header received
0806 00:52:49.071880 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:49.071896 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:49.072252 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:49.072626 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:49.073436 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.074453 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.075432 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.075904 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.076080 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:49.078970 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:49.078975 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:49.078987 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:49.079011 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:49.080785 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:49.081283 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:49.081287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:49.081291 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:49.081428 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:49.081818 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:49.081821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:49.081824 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:49.082331 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:49.082557 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:49.082809 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 00:52:49.083125 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":139,"threads_started":1}
I20260806 00:52:49.083899 8233 leader_election.cc:302] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57123e94c61548f6860746951767d8fd; no voters:
I20260806 00:52:49.084367 8233 leader_election.cc:288] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:52:49.084442 8240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:52:49.084720 8240 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd [term 1 LEADER]: Becoming Leader. State: Replica: 57123e94c61548f6860746951767d8fd, State: Running, Role: LEADER
I20260806 00:52:49.085096 8240 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57123e94c61548f6860746951767d8fd" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34673 } }
I20260806 00:52:49.085602 8233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:52:49.086781 8244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57123e94c61548f6860746951767d8fd. Latest consensus state: current_term: 1 leader_uuid: "57123e94c61548f6860746951767d8fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57123e94c61548f6860746951767d8fd" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34673 } } }
I20260806 00:52:49.086936 8244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd [sys.catalog]: This master's current role is: LEADER
I20260806 00:52:49.087278 8243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57123e94c61548f6860746951767d8fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57123e94c61548f6860746951767d8fd" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34673 } } }
I20260806 00:52:49.087380 8252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:52:49.087430 8243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd [sys.catalog]: This master's current role is: LEADER
I20260806 00:52:49.090385 8252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:52:49.095902 8252 catalog_manager.cc:1347] Generated new cluster ID: 482a149a579e422bb4178cdad0d59d74
I20260806 00:52:49.095968 8252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:52:49.123934 8252 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:52:49.125089 8252 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:52:49.133093 8252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd: Generated new TSK 0
I20260806 00:52:49.133893 8252 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
I20260806 00:52:49.165061 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:34673
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:40269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:52:49.283135 8274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:49.283388 8274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:49.283429 8274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:49.287933 8274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:49.287992 8274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:49.288048 8274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:52:49.293220 8274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:34673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:49.294240 8274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:49.295234 8274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:49.304841 8285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:49.304981 8274 server_base.cc:1034] running on GCE node
W20260806 00:52:49.304834 8281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:49.304834 8282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:49.305538 8274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:49.306281 8274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:49.307471 8274 hybrid_clock.cc:648] HybridClock initialized: now 1785977569307444 us; error 31 us; skew 500 ppm
Aug 06 00:52:49 dist-test-slave-45dx krb5kdc[8125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977569, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:49.310338 8274 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:52:49.311223 8274 webserver.cc:460] Webserver started at http://127.15.147.129:44299/ using document root <none> and password file <none>
I20260806 00:52:49.311717 8274 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:49.311774 8274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:49.311996 8274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:49.313969 8274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "6ffac27394aa4c74983bc3eaa639260b"
format_stamp: "Formatted at 2026-08-06 00:52:49 on dist-test-slave-45dx"
server_key: "736f4bcb35449d5420f3c4f6e0581593"
server_key_iv: "f395d5899470e9a52fe849fb3f410888"
server_key_version: "encryptionkey@0"
I20260806 00:52:49.314462 8274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "6ffac27394aa4c74983bc3eaa639260b"
format_stamp: "Formatted at 2026-08-06 00:52:49 on dist-test-slave-45dx"
server_key: "736f4bcb35449d5420f3c4f6e0581593"
server_key_iv: "f395d5899470e9a52fe849fb3f410888"
server_key_version: "encryptionkey@0"
I20260806 00:52:49.317981 8274 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:52:49.320166 8296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:49.321336 8274 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:52:49.321453 8274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "6ffac27394aa4c74983bc3eaa639260b"
format_stamp: "Formatted at 2026-08-06 00:52:49 on dist-test-slave-45dx"
server_key: "736f4bcb35449d5420f3c4f6e0581593"
server_key_iv: "f395d5899470e9a52fe849fb3f410888"
server_key_version: "encryptionkey@0"
I20260806 00:52:49.321554 8274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:49.342751 8274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:49.346127 8274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:49.346336 8274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:49.346952 8274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:49.347771 8274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:49.347842 8274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:49.347921 8274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:49.347954 8274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:49.357241 8274 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:40589
I20260806 00:52:49.357270 8452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:40589 every 8 connection(s)
I20260806 00:52:49.358101 8274 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:52:49.362787 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 8274
I20260806 00:52:49.362908 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:52:49.363229 15950 external_mini_cluster.cc:1428] Setting key 594561e11f6eb77e0ad9eedcca723fb9
Aug 06 00:52:49 dist-test-slave-45dx krb5kdc[8125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977569, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:52:49.373533 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:49.359770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:45027 (local address 127.15.147.190:34673)
0806 00:52:49.359964 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:49.359970 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:49.360834 (+ 864us) server_negotiation.cc:407] Connection header received
0806 00:52:49.362121 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:49.362124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:49.362169 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:49.362241 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:49.363569 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.364257 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.366058 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.366284 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.366370 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:49.369494 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:49.369497 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:49.369505 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:49.369533 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:49.371059 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:49.371732 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:49.371735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:49.371736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:49.371832 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:49.372360 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:49.372369 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:49.372370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:49.372552 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:49.372636 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:49.373293 (+ 657us) server_negotiation.cc:299] Negotiation successful
0806 00:52:49.373407 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 00:52:49.374724 8456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:49.360198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:34673 (local address 127.15.147.129:45027)
0806 00:52:49.360690 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:49.360732 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:49.361934 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:49.362401 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:49.362407 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:49.362760 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:49.363415 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:49.363431 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.364470 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.364474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:49.365922 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:49.365938 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.366446 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.366453 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:49.366700 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:49.367442 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:52:49.367458 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:49.369294 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:49.371181 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:49.371189 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:49.371205 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:49.371602 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:49.371941 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:49.371946 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:49.371951 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:49.372195 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:49.372669 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:49.372678 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:49.373014 (+ 336us) client_negotiation.cc:769] Sending connection context
0806 00:52:49.373246 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 00:52:49.373609 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":166,"threads_started":1}
WARNING: no policy specified for kudu/127.15.147.130@KRBTEST.COM; defaulting to no policy
I20260806 00:52:49.376109 8453 heartbeater.cc:347] Connected to a master server at 127.15.147.190:34673
I20260806 00:52:49.376401 8453 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:49.376915 8453 heartbeater.cc:511] Master 127.15.147.190:34673 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.130@KRBTEST.COM" created.
I20260806 00:52:49.378592 8190 ts_manager.cc:194] Registered new tserver with Master: 6ffac27394aa4c74983bc3eaa639260b (127.15.147.129:40589)
I20260806 00:52:49.379748 8190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:45027
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
I20260806 00:52:49.394624 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:34673
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:40269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:52:49.506816 8460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:49.507062 8460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:49.507138 8460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:49.511090 8460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:49.511155 8460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:49.511258 8460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:52:49.516283 8460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:34673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:49.517239 8460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:49.518275 8460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:49.527973 8471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:49.527984 8473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:49.527984 8470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:49.528023 8460 server_base.cc:1034] running on GCE node
I20260806 00:52:49.528666 8460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:49.529291 8460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:49.530475 8460 hybrid_clock.cc:648] HybridClock initialized: now 1785977569530441 us; error 44 us; skew 500 ppm
Aug 06 00:52:49 dist-test-slave-45dx krb5kdc[8125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977569, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:49.533354 8460 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:52:49.534233 8460 webserver.cc:460] Webserver started at http://127.15.147.130:44765/ using document root <none> and password file <none>
I20260806 00:52:49.534741 8460 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:49.534809 8460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:49.534968 8460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:49.537009 8460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "85eb7f58b536411f95d3a859e04aa8ac"
format_stamp: "Formatted at 2026-08-06 00:52:49 on dist-test-slave-45dx"
server_key: "c95eb6740b076717addd02108330b05c"
server_key_iv: "873d0e70fdc43634cd38bfd25e700b9b"
server_key_version: "encryptionkey@0"
I20260806 00:52:49.537513 8460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "85eb7f58b536411f95d3a859e04aa8ac"
format_stamp: "Formatted at 2026-08-06 00:52:49 on dist-test-slave-45dx"
server_key: "c95eb6740b076717addd02108330b05c"
server_key_iv: "873d0e70fdc43634cd38bfd25e700b9b"
server_key_version: "encryptionkey@0"
I20260806 00:52:49.541328 8460 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:52:49.543764 8484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:49.545073 8460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:52:49.545166 8460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "85eb7f58b536411f95d3a859e04aa8ac"
format_stamp: "Formatted at 2026-08-06 00:52:49 on dist-test-slave-45dx"
server_key: "c95eb6740b076717addd02108330b05c"
server_key_iv: "873d0e70fdc43634cd38bfd25e700b9b"
server_key_version: "encryptionkey@0"
I20260806 00:52:49.545251 8460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:49.562723 8460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:49.565877 8460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:49.566030 8460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:49.566527 8460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:49.567298 8460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:49.567346 8460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:49.567387 8460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:49.567409 8460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:49.577062 8460 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:44985
I20260806 00:52:49.577095 8659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:44985 every 8 connection(s)
I20260806 00:52:49.577893 8460 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:52:49.581458 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 8460
I20260806 00:52:49.581589 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:52:49.581952 15950 external_mini_cluster.cc:1428] Setting key e3749c5e212d4d3d87f7283aa91a9a76
Aug 06 00:52:49 dist-test-slave-45dx krb5kdc[8125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977569, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:52:49.593111 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:49.579569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:55921 (local address 127.15.147.190:34673)
0806 00:52:49.579726 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:49.579731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:49.580575 (+ 844us) server_negotiation.cc:407] Connection header received
0806 00:52:49.582230 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:49.582233 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:49.582281 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:49.582411 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:49.583695 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.584210 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.586016 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.586424 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.586574 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:49.589423 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:49.589426 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:49.589428 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:49.589452 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:49.590849 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:49.591499 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:49.591501 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:49.591502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:49.591576 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:49.591968 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:49.591970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:49.591971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:49.592143 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:49.592240 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:49.592895 (+ 655us) server_negotiation.cc:299] Negotiation successful
0806 00:52:49.592993 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
WARNING: no policy specified for kudu/127.15.147.131@KRBTEST.COM; defaulting to no policy
I20260806 00:52:49.594457 8662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:49.579955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:34673 (local address 127.15.147.130:55921)
0806 00:52:49.580439 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:49.580481 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:49.582009 (+ 1528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:49.582546 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:49.582553 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:49.582954 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:49.583522 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:49.583561 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.584364 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.584370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:49.585811 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:49.585827 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.586617 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.586621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:49.586762 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:49.587347 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:52:49.587369 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:49.589235 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:49.590984 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:49.590993 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:49.591010 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:49.591389 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:49.591665 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:49.591670 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:49.591672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:49.591868 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:49.592284 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:49.592292 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:49.592636 (+ 344us) client_negotiation.cc:769] Sending connection context
0806 00:52:49.592843 (+ 207us) client_negotiation.cc:240] Negotiation successful
0806 00:52:49.593197 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":114,"threads_started":1}
I20260806 00:52:49.595664 8660 heartbeater.cc:347] Connected to a master server at 127.15.147.190:34673
I20260806 00:52:49.595904 8660 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:49.596498 8660 heartbeater.cc:511] Master 127.15.147.190:34673 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.131@KRBTEST.COM" created.
I20260806 00:52:49.597705 8190 ts_manager.cc:194] Registered new tserver with Master: 85eb7f58b536411f95d3a859e04aa8ac (127.15.147.130:44985)
I20260806 00:52:49.598302 8190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:55921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
I20260806 00:52:49.612663 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:34673
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:40269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:52:49.725265 8667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:49.725535 8667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:49.725615 8667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:49.729723 8667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:49.729817 8667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:49.729907 8667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:52:49.735199 8667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:40269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:34673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:49.736236 8667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:49.737295 8667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:49.747272 8679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:49.747274 8672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:49.747339 8667 server_base.cc:1034] running on GCE node
W20260806 00:52:49.747445 8673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:49.747906 8667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:49.748622 8667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:49.749840 8667 hybrid_clock.cc:648] HybridClock initialized: now 1785977569749820 us; error 39 us; skew 500 ppm
Aug 06 00:52:49 dist-test-slave-45dx krb5kdc[8125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977569, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:49.752833 8667 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:52:49.753811 8667 webserver.cc:460] Webserver started at http://127.15.147.131:38379/ using document root <none> and password file <none>
I20260806 00:52:49.754400 8667 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:49.754490 8667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:49.754746 8667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:49.756909 8667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "fb78ff91e41d4be4b8c10b2ce1e59d92"
format_stamp: "Formatted at 2026-08-06 00:52:49 on dist-test-slave-45dx"
server_key: "ac25a49b7416f60ecf3771a63aba49c3"
server_key_iv: "5654f5d3a4685c0c2c7b19b4eda1aac8"
server_key_version: "encryptionkey@0"
I20260806 00:52:49.757474 8667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "fb78ff91e41d4be4b8c10b2ce1e59d92"
format_stamp: "Formatted at 2026-08-06 00:52:49 on dist-test-slave-45dx"
server_key: "ac25a49b7416f60ecf3771a63aba49c3"
server_key_iv: "5654f5d3a4685c0c2c7b19b4eda1aac8"
server_key_version: "encryptionkey@0"
I20260806 00:52:49.761252 8667 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:52:49.763590 8692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:49.764803 8667 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:52:49.764966 8667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "fb78ff91e41d4be4b8c10b2ce1e59d92"
format_stamp: "Formatted at 2026-08-06 00:52:49 on dist-test-slave-45dx"
server_key: "ac25a49b7416f60ecf3771a63aba49c3"
server_key_iv: "5654f5d3a4685c0c2c7b19b4eda1aac8"
server_key_version: "encryptionkey@0"
I20260806 00:52:49.765081 8667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:49.793586 8667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:49.797169 8667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:49.797391 8667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:49.798084 8667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:49.799000 8667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:49.799079 8667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:49.799152 8667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:49.799196 8667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:49.808633 8667 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:44513
I20260806 00:52:49.808677 8843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:44513 every 8 connection(s)
I20260806 00:52:49.809613 8667 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:52:49.819468 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 8667
I20260806 00:52:49.819650 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
Aug 06 00:52:49 dist-test-slave-45dx krb5kdc[8125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977569, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:52:49.819991 15950 external_mini_cluster.cc:1428] Setting key 860f8eb15e3cdc24e51d5b8c109063e9
I20260806 00:52:49.824187 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:49.811473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:60855 (local address 127.15.147.190:34673)
0806 00:52:49.811622 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:49.811626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:49.812527 (+ 901us) server_negotiation.cc:407] Connection header received
0806 00:52:49.813977 (+ 1450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:49.813981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:49.814045 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:49.814162 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:49.815479 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.816028 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.817328 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.817560 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.817678 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:49.820452 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:49.820458 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:49.820463 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:49.820497 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:49.822197 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:49.822733 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:49.822735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:49.822736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:49.822816 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:49.823124 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:49.823127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:49.823128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:49.823299 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:49.823375 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:49.823827 (+ 452us) server_negotiation.cc:299] Negotiation successful
0806 00:52:49.823979 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:52:49.825335 8846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:49.811839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:34673 (local address 127.15.147.131:60855)
0806 00:52:49.812344 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:49.812427 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:49.813742 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:49.814374 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:49.814381 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:49.814768 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:49.815288 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:49.815304 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.816181 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.816184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:49.817183 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:49.817195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.817743 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.817748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:49.817931 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:49.818552 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:52:49.818570 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:49.820238 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:49.822317 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:49.822323 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:49.822338 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:49.822632 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:49.822898 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:49.822900 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:49.822901 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:49.823039 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:49.823385 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:49.823390 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:49.823620 (+ 230us) client_negotiation.cc:769] Sending connection context
0806 00:52:49.823766 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 00:52:49.824134 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":111,"threads_started":1}
I20260806 00:52:49.826668 8844 heartbeater.cc:347] Connected to a master server at 127.15.147.190:34673
I20260806 00:52:49.826924 8844 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:49.827520 8844 heartbeater.cc:511] Master 127.15.147.190:34673 requested a full tablet report, sending...
I20260806 00:52:49.828594 8190 ts_manager.cc:194] Registered new tserver with Master: fb78ff91e41d4be4b8c10b2ce1e59d92 (127.15.147.131:44513)
I20260806 00:52:49.829193 8190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:60855
I20260806 00:52:49.833984 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:52:49.843457 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:49.836007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44690 (local address 127.15.147.190:34673)
0806 00:52:49.836238 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:49.836242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:49.836367 (+ 125us) server_negotiation.cc:407] Connection header received
0806 00:52:49.836522 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:49.836524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:49.836563 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:49.836631 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:49.837272 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.837759 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.838610 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.838823 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.838942 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:49.840379 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:49.840382 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:49.840384 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:49.840405 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:49.841959 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:49.842343 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:49.842346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:49.842347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:49.842433 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:49.842696 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:49.842699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:49.842700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:49.842941 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:49.843052 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:49.843248 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 00:52:49.843343 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260806 00:52:49.846282 8190 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:52:49.849428 8190 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:52:49.863848 8863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:49.858031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56360 (local address 127.15.147.130:44985)
0806 00:52:49.859212 (+ 1181us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:49.859227 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:49.859250 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:52:49.859324 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:49.859330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:49.859516 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:49.859940 (+ 424us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:49.860065 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.860886 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.862590 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.863144 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.863283 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:49.863432 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:49.863590 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 00:52:49.863693 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":945,"thread_start_us":126,"threads_started":1}
I20260806 00:52:49.863839 8861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:49.857509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:44985 (local address 127.0.0.1:56360)
0806 00:52:49.858188 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:49.858205 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:49.858442 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:49.859687 (+ 1245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:49.859690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:49.859706 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:49.859796 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:49.859804 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.861043 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.861049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:49.862465 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:49.862473 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.863272 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.863275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:49.863381 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:49.863419 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 00:52:49.863560 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 00:52:49.863697 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":205,"thread_start_us":94,"threads_started":1}
I20260806 00:52:49.864099 8864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:49.858094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:44513 (local address 127.0.0.1:59954)
0806 00:52:49.858759 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:49.858771 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:49.859112 (+ 341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:49.859596 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:49.859602 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:49.859625 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:49.859733 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:49.859740 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.861100 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.861103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:49.862229 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:49.862239 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.863550 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.863554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:49.863749 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:49.863810 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 00:52:49.863887 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 00:52:49.863965 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":132,"thread_start_us":112,"threads_started":1}
I20260806 00:52:49.864224 8862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:49.857542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32926 (local address 127.15.147.129:40589)
0806 00:52:49.857799 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:49.857821 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:49.858214 (+ 393us) server_negotiation.cc:407] Connection header received
0806 00:52:49.858549 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:49.858555 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:49.858774 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:49.858915 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:49.859445 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.860696 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.862486 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.863275 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.863382 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:49.863482 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:49.863966 (+ 484us) server_negotiation.cc:299] Negotiation successful
0806 00:52:49.864070 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":73,"threads_started":1}
I20260806 00:52:49.864569 8859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:49.857395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:40589 (local address 127.0.0.1:32926)
0806 00:52:49.858054 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:49.858108 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:49.858329 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:49.859138 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:49.859146 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:49.859170 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:49.859311 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:49.859318 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.860840 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.860843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:49.862288 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:49.862299 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.863550 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.863554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:49.863785 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:49.863823 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 00:52:49.864390 (+ 567us) client_negotiation.cc:240] Negotiation successful
0806 00:52:49.864459 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":113,"thread_start_us":82,"threads_started":1}
I20260806 00:52:49.864576 8865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:49.858561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59954 (local address 127.15.147.131:44513)
0806 00:52:49.859148 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:49.859171 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:49.859196 (+ 25us) server_negotiation.cc:407] Connection header received
0806 00:52:49.859271 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:49.859275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:49.859443 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:49.859596 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:49.859866 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.860942 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.862361 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:49.863202 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:49.863343 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:49.863482 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:49.864278 (+ 796us) server_negotiation.cc:299] Negotiation successful
0806 00:52:49.864432 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":165,"threads_started":1}
I20260806 00:52:49.867327 8366 tablet_service.cc:1467] Processing CreateTablet for tablet 73697a64091d49a7b0d023e383a60eee (DEFAULT_TABLE table=test-table [id=886317ba806846bea847696d972d89d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:49.867475 8753 tablet_service.cc:1467] Processing CreateTablet for tablet 73697a64091d49a7b0d023e383a60eee (DEFAULT_TABLE table=test-table [id=886317ba806846bea847696d972d89d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:49.867594 8559 tablet_service.cc:1467] Processing CreateTablet for tablet 73697a64091d49a7b0d023e383a60eee (DEFAULT_TABLE table=test-table [id=886317ba806846bea847696d972d89d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:49.868409 8366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73697a64091d49a7b0d023e383a60eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:49.868489 8559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73697a64091d49a7b0d023e383a60eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:49.868489 8753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73697a64091d49a7b0d023e383a60eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:49.875054 8871 tablet_bootstrap.cc:492] T 73697a64091d49a7b0d023e383a60eee P 6ffac27394aa4c74983bc3eaa639260b: Bootstrap starting.
I20260806 00:52:49.876345 8874 tablet_bootstrap.cc:492] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92: Bootstrap starting.
I20260806 00:52:49.876523 8873 tablet_bootstrap.cc:492] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac: Bootstrap starting.
I20260806 00:52:49.877291 8871 tablet_bootstrap.cc:654] T 73697a64091d49a7b0d023e383a60eee P 6ffac27394aa4c74983bc3eaa639260b: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:49.878018 8871 log.cc:824] T 73697a64091d49a7b0d023e383a60eee P 6ffac27394aa4c74983bc3eaa639260b: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:49.878433 8873 tablet_bootstrap.cc:654] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:49.878432 8874 tablet_bootstrap.cc:654] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:49.879778 8874 log.cc:824] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:49.879778 8873 log.cc:824] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:49.879948 8871 tablet_bootstrap.cc:492] T 73697a64091d49a7b0d023e383a60eee P 6ffac27394aa4c74983bc3eaa639260b: No bootstrap required, opened a new log
I20260806 00:52:49.880208 8871 ts_tablet_manager.cc:1397] T 73697a64091d49a7b0d023e383a60eee P 6ffac27394aa4c74983bc3eaa639260b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:52:49.881541 8874 tablet_bootstrap.cc:492] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92: No bootstrap required, opened a new log
I20260806 00:52:49.881768 8874 ts_tablet_manager.cc:1397] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:52:49.881938 8873 tablet_bootstrap.cc:492] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac: No bootstrap required, opened a new log
I20260806 00:52:49.882153 8873 ts_tablet_manager.cc:1397] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:52:49.884685 8874 raft_consensus.cc:348] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffac27394aa4c74983bc3eaa639260b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40589 } } peers { permanent_uuid: "85eb7f58b536411f95d3a859e04aa8ac" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44985 } } peers { permanent_uuid: "fb78ff91e41d4be4b8c10b2ce1e59d92" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44513 } }
I20260806 00:52:49.884724 8871 raft_consensus.cc:348] T 73697a64091d49a7b0d023e383a60eee P 6ffac27394aa4c74983bc3eaa639260b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffac27394aa4c74983bc3eaa639260b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40589 } } peers { permanent_uuid: "85eb7f58b536411f95d3a859e04aa8ac" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44985 } } peers { permanent_uuid: "fb78ff91e41d4be4b8c10b2ce1e59d92" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44513 } }
I20260806 00:52:49.884924 8874 raft_consensus.cc:374] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:49.884984 8874 raft_consensus.cc:729] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb78ff91e41d4be4b8c10b2ce1e59d92, State: Initialized, Role: FOLLOWER
I20260806 00:52:49.885020 8871 raft_consensus.cc:374] T 73697a64091d49a7b0d023e383a60eee P 6ffac27394aa4c74983bc3eaa639260b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:49.885111 8871 raft_consensus.cc:729] T 73697a64091d49a7b0d023e383a60eee P 6ffac27394aa4c74983bc3eaa639260b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ffac27394aa4c74983bc3eaa639260b, State: Initialized, Role: FOLLOWER
I20260806 00:52:49.885490 8874 consensus_queue.cc:260] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffac27394aa4c74983bc3eaa639260b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40589 } } peers { permanent_uuid: "85eb7f58b536411f95d3a859e04aa8ac" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44985 } } peers { permanent_uuid: "fb78ff91e41d4be4b8c10b2ce1e59d92" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44513 } }
I20260806 00:52:49.885577 8871 consensus_queue.cc:260] T 73697a64091d49a7b0d023e383a60eee P 6ffac27394aa4c74983bc3eaa639260b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffac27394aa4c74983bc3eaa639260b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40589 } } peers { permanent_uuid: "85eb7f58b536411f95d3a859e04aa8ac" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44985 } } peers { permanent_uuid: "fb78ff91e41d4be4b8c10b2ce1e59d92" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44513 } }
I20260806 00:52:49.886215 8873 raft_consensus.cc:348] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffac27394aa4c74983bc3eaa639260b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40589 } } peers { permanent_uuid: "85eb7f58b536411f95d3a859e04aa8ac" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44985 } } peers { permanent_uuid: "fb78ff91e41d4be4b8c10b2ce1e59d92" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44513 } }
I20260806 00:52:49.886452 8873 raft_consensus.cc:374] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:49.886557 8873 raft_consensus.cc:729] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85eb7f58b536411f95d3a859e04aa8ac, State: Initialized, Role: FOLLOWER
I20260806 00:52:49.886718 8874 ts_tablet_manager.cc:1428] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 00:52:49.886747 8871 ts_tablet_manager.cc:1428] T 73697a64091d49a7b0d023e383a60eee P 6ffac27394aa4c74983bc3eaa639260b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 00:52:49.887086 8453 heartbeater.cc:503] Master 127.15.147.190:34673 was elected leader, sending a full tablet report...
I20260806 00:52:49.887084 8844 heartbeater.cc:503] Master 127.15.147.190:34673 was elected leader, sending a full tablet report...
I20260806 00:52:49.887022 8873 consensus_queue.cc:260] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffac27394aa4c74983bc3eaa639260b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40589 } } peers { permanent_uuid: "85eb7f58b536411f95d3a859e04aa8ac" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44985 } } peers { permanent_uuid: "fb78ff91e41d4be4b8c10b2ce1e59d92" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44513 } }
I20260806 00:52:49.887681 8660 heartbeater.cc:503] Master 127.15.147.190:34673 was elected leader, sending a full tablet report...
I20260806 00:52:49.887928 8873 ts_tablet_manager.cc:1428] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:52:50.049290 8883 raft_consensus.cc:482] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:52:50.049501 8883 raft_consensus.cc:504] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffac27394aa4c74983bc3eaa639260b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40589 } } peers { permanent_uuid: "85eb7f58b536411f95d3a859e04aa8ac" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44985 } } peers { permanent_uuid: "fb78ff91e41d4be4b8c10b2ce1e59d92" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44513 } }
I20260806 00:52:50.050709 8883 leader_election.cc:288] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ffac27394aa4c74983bc3eaa639260b (127.15.147.129:40589), fb78ff91e41d4be4b8c10b2ce1e59d92 (127.15.147.131:44513)
I20260806 00:52:50.054350 8862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:50.051063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:41305 (local address 127.15.147.129:40589)
0806 00:52:50.051188 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:50.051193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:50.051208 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:52:50.051371 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:50.051374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:50.051428 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:50.051512 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:50.051773 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.052285 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.053389 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.053767 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.053923 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:50.053978 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:50.054157 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 00:52:50.054222 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 00:52:50.054350 8662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:50.050933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:40589 (local address 127.15.147.130:41305)
0806 00:52:50.051129 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:50.051146 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:50.051241 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:50.051527 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:50.051530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:50.051547 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:50.051666 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:50.051671 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.052457 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.052460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:50.053261 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:50.053270 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.053912 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.053916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:50.054025 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:50.054050 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 00:52:50.054141 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 00:52:50.054216 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":1}
I20260806 00:52:50.054600 8865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:50.051140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:38231 (local address 127.15.147.131:44513)
0806 00:52:50.051274 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:50.051279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:50.051470 (+ 191us) server_negotiation.cc:407] Connection header received
0806 00:52:50.051627 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:50.051630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:50.051676 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:50.051765 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:50.052061 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.052603 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.053629 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.054046 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.054178 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:50.054228 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:50.054418 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 00:52:50.054494 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:52:50.054629 8884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:50.050969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:44513 (local address 127.15.147.130:38231)
0806 00:52:50.051365 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:50.051375 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:50.051503 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:50.051776 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:50.051779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:50.051794 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:50.051941 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:50.051946 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.052724 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.052729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:50.053500 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:50.053509 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.054174 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.054178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:50.054278 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:50.054306 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 00:52:50.054399 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 00:52:50.054470 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":110,"threads_started":1}
I20260806 00:52:50.055136 8391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73697a64091d49a7b0d023e383a60eee" candidate_uuid: "85eb7f58b536411f95d3a859e04aa8ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ffac27394aa4c74983bc3eaa639260b" is_pre_election: true
I20260806 00:52:50.055169 8785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73697a64091d49a7b0d023e383a60eee" candidate_uuid: "85eb7f58b536411f95d3a859e04aa8ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb78ff91e41d4be4b8c10b2ce1e59d92" is_pre_election: true
I20260806 00:52:50.055440 8391 raft_consensus.cc:2454] T 73697a64091d49a7b0d023e383a60eee P 6ffac27394aa4c74983bc3eaa639260b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85eb7f58b536411f95d3a859e04aa8ac in term 0.
I20260806 00:52:50.055459 8785 raft_consensus.cc:2454] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85eb7f58b536411f95d3a859e04aa8ac in term 0.
I20260806 00:52:50.055922 8487 leader_election.cc:302] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85eb7f58b536411f95d3a859e04aa8ac, fb78ff91e41d4be4b8c10b2ce1e59d92; no voters:
I20260806 00:52:50.056207 8883 raft_consensus.cc:2781] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:52:50.056288 8883 raft_consensus.cc:482] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:52:50.056382 8883 raft_consensus.cc:3037] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:50.057507 8883 raft_consensus.cc:504] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffac27394aa4c74983bc3eaa639260b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40589 } } peers { permanent_uuid: "85eb7f58b536411f95d3a859e04aa8ac" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44985 } } peers { permanent_uuid: "fb78ff91e41d4be4b8c10b2ce1e59d92" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44513 } }
I20260806 00:52:50.057916 8883 leader_election.cc:288] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [CANDIDATE]: Term 1 election: Requested vote from peers 6ffac27394aa4c74983bc3eaa639260b (127.15.147.129:40589), fb78ff91e41d4be4b8c10b2ce1e59d92 (127.15.147.131:44513)
I20260806 00:52:50.058350 8391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73697a64091d49a7b0d023e383a60eee" candidate_uuid: "85eb7f58b536411f95d3a859e04aa8ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ffac27394aa4c74983bc3eaa639260b"
I20260806 00:52:50.058466 8785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73697a64091d49a7b0d023e383a60eee" candidate_uuid: "85eb7f58b536411f95d3a859e04aa8ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb78ff91e41d4be4b8c10b2ce1e59d92"
I20260806 00:52:50.058507 8391 raft_consensus.cc:3037] T 73697a64091d49a7b0d023e383a60eee P 6ffac27394aa4c74983bc3eaa639260b [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:50.058625 8785 raft_consensus.cc:3037] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:50.059671 8391 raft_consensus.cc:2454] T 73697a64091d49a7b0d023e383a60eee P 6ffac27394aa4c74983bc3eaa639260b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85eb7f58b536411f95d3a859e04aa8ac in term 1.
I20260806 00:52:50.059810 8785 raft_consensus.cc:2454] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85eb7f58b536411f95d3a859e04aa8ac in term 1.
I20260806 00:52:50.060048 8485 leader_election.cc:302] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ffac27394aa4c74983bc3eaa639260b, 85eb7f58b536411f95d3a859e04aa8ac; no voters:
I20260806 00:52:50.060249 8883 raft_consensus.cc:2781] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [term 1 FOLLOWER]: Leader election won for term 1
W20260806 00:52:50.060496 8845 tablet.cc:2369] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:52:50.060568 8883 raft_consensus.cc:686] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [term 1 LEADER]: Becoming Leader. State: Replica: 85eb7f58b536411f95d3a859e04aa8ac, State: Running, Role: LEADER
I20260806 00:52:50.060920 8883 consensus_queue.cc:237] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffac27394aa4c74983bc3eaa639260b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40589 } } peers { permanent_uuid: "85eb7f58b536411f95d3a859e04aa8ac" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44985 } } peers { permanent_uuid: "fb78ff91e41d4be4b8c10b2ce1e59d92" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44513 } }
I20260806 00:52:50.064404 8190 catalog_manager.cc:5495] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 85eb7f58b536411f95d3a859e04aa8ac (127.15.147.130). New cstate: current_term: 1 leader_uuid: "85eb7f58b536411f95d3a859e04aa8ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffac27394aa4c74983bc3eaa639260b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85eb7f58b536411f95d3a859e04aa8ac" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb78ff91e41d4be4b8c10b2ce1e59d92" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44513 } health_report { overall_health: UNKNOWN } } }
W20260806 00:52:50.079022 8661 tablet.cc:2369] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:52:50.096758 8863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:50.093632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56366 (local address 127.15.147.130:44985)
0806 00:52:50.093753 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:50.093757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:50.093770 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:52:50.093834 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:50.093836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:50.093904 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:50.093987 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:52:50.094189 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.094659 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.095468 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.095670 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.095743 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:50.096000 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:52:50.096438 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:52:50.096525 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:50.096591 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 00:52:50.096639 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 00:52:50.102593 8391 raft_consensus.cc:1261] T 73697a64091d49a7b0d023e383a60eee P 6ffac27394aa4c74983bc3eaa639260b [term 1 FOLLOWER]: Refusing update from remote peer 85eb7f58b536411f95d3a859e04aa8ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:52:50.102593 8785 raft_consensus.cc:1261] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92 [term 1 FOLLOWER]: Refusing update from remote peer 85eb7f58b536411f95d3a859e04aa8ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:52:50.103304 8883 consensus_queue.cc:1035] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb78ff91e41d4be4b8c10b2ce1e59d92" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 44513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:52:50.103493 8887 consensus_queue.cc:1035] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ffac27394aa4c74983bc3eaa639260b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 00:52:50.109385 8455 tablet.cc:2369] T 73697a64091d49a7b0d023e383a60eee P 6ffac27394aa4c74983bc3eaa639260b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:52:50.110201 8891 mvcc.cc:205] Tried to move back new op lower bound from 7315364127132176384 to 7315364126972243968. Current Snapshot: MvccSnapshot[applied={T|T < 7315364127132176384}]
I20260806 00:52:50.112739 8893 mvcc.cc:205] Tried to move back new op lower bound from 7315364127132176384 to 7315364126972243968. Current Snapshot: MvccSnapshot[applied={T|T < 7315364127132176384}]
I20260806 00:52:50.113768 8894 mvcc.cc:205] Tried to move back new op lower bound from 7315364127132176384 to 7315364126972243968. Current Snapshot: MvccSnapshot[applied={T|T < 7315364127132176384}]
I20260806 00:52:50.118939 8189 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:52:50.119124 8189 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:52:50.121609 8189 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57123e94c61548f6860746951767d8fd: Sending DeleteTablet for 3 replicas of tablet 73697a64091d49a7b0d023e383a60eee
I20260806 00:52:50.122448 8559 tablet_service.cc:1514] Processing DeleteTablet for tablet 73697a64091d49a7b0d023e383a60eee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:50 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:56360
I20260806 00:52:50.122457 8753 tablet_service.cc:1514] Processing DeleteTablet for tablet 73697a64091d49a7b0d023e383a60eee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:50 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:59954
I20260806 00:52:50.122952 8908 tablet_replica.cc:330] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92: stopping tablet replica
I20260806 00:52:50.123083 8366 tablet_service.cc:1514] Processing DeleteTablet for tablet 73697a64091d49a7b0d023e383a60eee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:50 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:32926
I20260806 00:52:50.123342 8908 raft_consensus.cc:2229] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:52:50.123652 8908 raft_consensus.cc:2258] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:50.123749 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 8274
I20260806 00:52:50.130422 8908 ts_tablet_manager.cc:1905] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:52:50.131942 8909 tablet_replica.cc:330] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac: stopping tablet replica
I20260806 00:52:50.132248 8909 raft_consensus.cc:2229] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:52:50.132656 8909 raft_consensus.cc:2258] T 73697a64091d49a7b0d023e383a60eee P 85eb7f58b536411f95d3a859e04aa8ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:50.133450 8908 ts_tablet_manager.cc:1918] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:52:50.133563 8908 log.cc:1196] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/73697a64091d49a7b0d023e383a60eee
I20260806 00:52:50.133913 8908 ts_tablet_manager.cc:1939] T 73697a64091d49a7b0d023e383a60eee P fb78ff91e41d4be4b8c10b2ce1e59d92: Deleting consensus metadata
I20260806 00:52:50.135215 8170 catalog_manager.cc:4841] TS fb78ff91e41d4be4b8c10b2ce1e59d92 (127.15.147.131:44513): tablet 73697a64091d49a7b0d023e383a60eee (table test-table [id=886317ba806846bea847696d972d89d1]) successfully deleted
W20260806 00:52:50.139581 8168 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.147.129:40589 (error 108)
I20260806 00:52:50.139734 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 8460
I20260806 00:52:50.147917 8859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:50.146942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:40589 (local address 127.0.0.1:32940)
0806 00:52:50.147731 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:50.147818 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":676}
W20260806 00:52:50.148168 8168 catalog_manager.cc:4584] TS 6ffac27394aa4c74983bc3eaa639260b (127.15.147.129:40589): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 73697a64091d49a7b0d023e383a60eee: Network error: Client connection negotiation failed: client connection to 127.15.147.129:40589: connect: Connection refused (error 111)
I20260806 00:52:50.153823 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 8667
I20260806 00:52:50.157979 8859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:50.153944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:44985 (local address 127.0.0.1:56372)
0806 00:52:50.157783 (+ 3839us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:50.157867 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.130:44985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3726}
I20260806 00:52:50.163125 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 8143
2026-08-06T00:52:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2618 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T00:52:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:52:50Z Disabled control of system clock
I20260806 00:52:50.200695 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:39909
--webserver_interface=127.15.147.190
--webserver_port=0
--builtin_ntp_servers=127.15.147.148:35903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:39909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 00:52:50.312209 8915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:50.312532 8915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:50.312567 8915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:50.316425 8915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:52:50.316469 8915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:50.316483 8915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:50.316497 8915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:52:50.316510 8915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:52:50.321030 8915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:35903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:39909
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:39909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:50.322005 8915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:50.323258 8915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:50.332070 8921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:50.332226 8915 server_base.cc:1034] running on GCE node
W20260806 00:52:50.332082 8923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:50.332052 8920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:50.332919 8915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:50.334059 8915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:50.335273 8915 hybrid_clock.cc:648] HybridClock initialized: now 1785977570335246 us; error 39 us; skew 500 ppm
I20260806 00:52:50.337358 8915 webserver.cc:460] Webserver started at http://127.15.147.190:37287/ using document root <none> and password file <none>
I20260806 00:52:50.337966 8915 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:50.338063 8915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:50.338307 8915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:50.340476 8915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "0873a0c4176d40c789cdee0b8f5d73b1"
format_stamp: "Formatted at 2026-08-06 00:52:50 on dist-test-slave-45dx"
server_key: "1d6a500e376e984f45335dcd31831d4b"
server_key_iv: "960b88197fc44e8bcc8eac0bada700f4"
server_key_version: "encryptionkey@0"
I20260806 00:52:50.341048 8915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "0873a0c4176d40c789cdee0b8f5d73b1"
format_stamp: "Formatted at 2026-08-06 00:52:50 on dist-test-slave-45dx"
server_key: "1d6a500e376e984f45335dcd31831d4b"
server_key_iv: "960b88197fc44e8bcc8eac0bada700f4"
server_key_version: "encryptionkey@0"
I20260806 00:52:50.345351 8915 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:52:50.347734 8930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:50.349071 8915 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:52:50.349220 8915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "0873a0c4176d40c789cdee0b8f5d73b1"
format_stamp: "Formatted at 2026-08-06 00:52:50 on dist-test-slave-45dx"
server_key: "1d6a500e376e984f45335dcd31831d4b"
server_key_iv: "960b88197fc44e8bcc8eac0bada700f4"
server_key_version: "encryptionkey@0"
I20260806 00:52:50.349330 8915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:50.386129 8915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:50.386945 8915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:50.387162 8915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:50.394431 8915 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:39909
I20260806 00:52:50.394448 9009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:39909 every 8 connection(s)
I20260806 00:52:50.395438 8915 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:52:50.397491 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 8915
I20260806 00:52:50.397629 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:52:50.397987 15950 external_mini_cluster.cc:1428] Setting key 37407a241d44b2656f1977e71ba93761
I20260806 00:52:50.399178 9010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:50.405858 9010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1: Bootstrap starting.
I20260806 00:52:50.407373 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:50.399453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35378 (local address 127.15.147.190:39909)
0806 00:52:50.400045 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:50.400061 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:50.400095 (+ 34us) server_negotiation.cc:407] Connection header received
0806 00:52:50.400751 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:50.400766 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:50.401120 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:50.401383 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:50.401713 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.402737 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.403962 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.404461 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.404579 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:50.405601 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:50.405606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:50.405617 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:52:50.406032 (+ 415us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:50.406105 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:52:50.406126 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:50.406200 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:50.406368 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 00:52:50.406665 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":141,"threads_started":1}
I20260806 00:52:50.408501 9010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:50.409219 9010 log.cc:824] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:50.411051 9010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1: No bootstrap required, opened a new log
I20260806 00:52:50.414395 9010 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0873a0c4176d40c789cdee0b8f5d73b1" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39909 } }
I20260806 00:52:50.414638 9010 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:50.414736 9010 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0873a0c4176d40c789cdee0b8f5d73b1, State: Initialized, Role: FOLLOWER
I20260806 00:52:50.415122 9010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0873a0c4176d40c789cdee0b8f5d73b1" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39909 } }
I20260806 00:52:50.415287 9010 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:52:50.415369 9010 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:52:50.415495 9010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:50.416421 9010 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0873a0c4176d40c789cdee0b8f5d73b1" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39909 } }
I20260806 00:52:50.416742 9010 leader_election.cc:302] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0873a0c4176d40c789cdee0b8f5d73b1; no voters:
I20260806 00:52:50.417042 9010 leader_election.cc:288] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:52:50.417181 9017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:52:50.417443 9017 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1 [term 1 LEADER]: Becoming Leader. State: Replica: 0873a0c4176d40c789cdee0b8f5d73b1, State: Running, Role: LEADER
I20260806 00:52:50.417825 9017 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0873a0c4176d40c789cdee0b8f5d73b1" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39909 } }
I20260806 00:52:50.418195 9010 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:52:50.419484 9018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0873a0c4176d40c789cdee0b8f5d73b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0873a0c4176d40c789cdee0b8f5d73b1" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39909 } } }
I20260806 00:52:50.419487 9019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0873a0c4176d40c789cdee0b8f5d73b1. Latest consensus state: current_term: 1 leader_uuid: "0873a0c4176d40c789cdee0b8f5d73b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0873a0c4176d40c789cdee0b8f5d73b1" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 39909 } } }
I20260806 00:52:50.419646 9018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1 [sys.catalog]: This master's current role is: LEADER
I20260806 00:52:50.419646 9019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1 [sys.catalog]: This master's current role is: LEADER
I20260806 00:52:50.420110 9031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:52:50.422827 9031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:52:50.427810 9031 catalog_manager.cc:1347] Generated new cluster ID: a7b01a49ef374b029ec45159bf959c52
I20260806 00:52:50.427868 9031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:52:50.447767 9031 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:52:50.448717 9031 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:52:50.465891 9031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1: Generated new TSK 0
I20260806 00:52:50.466687 9031 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:52:50.480693 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39909
--builtin_ntp_servers=127.15.147.148:35903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 00:52:50.610963 9040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:50.611243 9040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:50.611284 9040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:50.615406 9040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:50.615463 9040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:50.615517 9040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:52:50.620604 9040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:35903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:50.621732 9040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:50.623204 9040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:50.632966 9052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:50.632998 9051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:50.632997 9057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:50.633064 9040 server_base.cc:1034] running on GCE node
I20260806 00:52:50.633606 9040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:50.634223 9040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:50.635421 9040 hybrid_clock.cc:648] HybridClock initialized: now 1785977570635407 us; error 32 us; skew 500 ppm
I20260806 00:52:50.637348 9040 webserver.cc:460] Webserver started at http://127.15.147.129:46141/ using document root <none> and password file <none>
I20260806 00:52:50.637948 9040 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:50.638048 9040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:50.638288 9040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:50.640376 9040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "4355a584422f47e294e54ab2dc1c4c4e"
format_stamp: "Formatted at 2026-08-06 00:52:50 on dist-test-slave-45dx"
server_key: "15520bc8e43e23c4f6b7a737c6272167"
server_key_iv: "3b8033ccbcd26e4c4445262a1e2f96ba"
server_key_version: "encryptionkey@0"
I20260806 00:52:50.640978 9040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "4355a584422f47e294e54ab2dc1c4c4e"
format_stamp: "Formatted at 2026-08-06 00:52:50 on dist-test-slave-45dx"
server_key: "15520bc8e43e23c4f6b7a737c6272167"
server_key_iv: "3b8033ccbcd26e4c4445262a1e2f96ba"
server_key_version: "encryptionkey@0"
I20260806 00:52:50.644577 9040 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:52:50.646929 9066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:50.647989 9040 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:52:50.648121 9040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "4355a584422f47e294e54ab2dc1c4c4e"
format_stamp: "Formatted at 2026-08-06 00:52:50 on dist-test-slave-45dx"
server_key: "15520bc8e43e23c4f6b7a737c6272167"
server_key_iv: "3b8033ccbcd26e4c4445262a1e2f96ba"
server_key_version: "encryptionkey@0"
I20260806 00:52:50.648228 9040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:50.668826 9040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:50.669477 9040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:50.669675 9040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:50.670260 9040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:50.671178 9040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:50.671250 9040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:50.671321 9040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:50.671365 9040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:50.680270 9040 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:46235
I20260806 00:52:50.680344 9228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:46235 every 8 connection(s)
I20260806 00:52:50.681166 9040 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:52:50.689356 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 9040
I20260806 00:52:50.689522 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:52:50.689881 15950 external_mini_cluster.cc:1428] Setting key 3f7821e2ce1409eedc9d8d1dec0d0b4d
I20260806 00:52:50.690632 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:50.683052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:60625 (local address 127.15.147.190:39909)
0806 00:52:50.683213 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:50.683217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:50.684226 (+ 1009us) server_negotiation.cc:407] Connection header received
0806 00:52:50.685721 (+ 1495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:50.685725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:50.685789 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:50.685866 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:50.686728 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.687304 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.688480 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.688721 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.688832 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:50.690045 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:50.690055 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:50.690058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:52:50.690084 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:50.690104 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:52:50.690112 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:50.690172 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:50.690391 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 00:52:50.690473 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 00:52:50.691517 9235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:50.683435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:39909 (local address 127.15.147.129:60625)
0806 00:52:50.684066 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:50.684105 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:50.685500 (+ 1395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:50.686050 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:50.686056 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:50.686135 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:50.686582 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:50.686593 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.687451 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.687457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:50.688347 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:50.688357 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.688897 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.688904 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:50.689058 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:50.689707 (+ 649us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:52:50.689709 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:50.689764 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:52:50.689841 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:50.690206 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:50.690216 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:50.690222 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 00:52:50.690362 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 00:52:50.690622 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":116,"threads_started":1}
I20260806 00:52:50.692144 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39909
--builtin_ntp_servers=127.15.147.148:35903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 00:52:50.692850 9232 heartbeater.cc:347] Connected to a master server at 127.15.147.190:39909
I20260806 00:52:50.693089 9232 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:50.693637 9232 heartbeater.cc:511] Master 127.15.147.190:39909 requested a full tablet report, sending...
I20260806 00:52:50.695163 8955 ts_manager.cc:194] Registered new tserver with Master: 4355a584422f47e294e54ab2dc1c4c4e (127.15.147.129:46235)
I20260806 00:52:50.696267 8955 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.147.129:60625
W20260806 00:52:50.807662 9236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:50.807909 9236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:50.807946 9236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:50.811991 9236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:50.812044 9236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:50.812127 9236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:52:50.816836 9236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:35903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:50.817883 9236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:50.819423 9236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:50.829793 9245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:50.830173 9250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:50.829891 9236 server_base.cc:1034] running on GCE node
W20260806 00:52:50.830214 9247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:50.831532 9236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:50.832212 9236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:50.833446 9236 hybrid_clock.cc:648] HybridClock initialized: now 1785977570833424 us; error 35 us; skew 500 ppm
I20260806 00:52:50.835520 9236 webserver.cc:460] Webserver started at http://127.15.147.130:43473/ using document root <none> and password file <none>
I20260806 00:52:50.836162 9236 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:50.836264 9236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:50.836535 9236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:50.838735 9236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "859ff4d702b947369b9fe282ed78315f"
format_stamp: "Formatted at 2026-08-06 00:52:50 on dist-test-slave-45dx"
server_key: "bfad9c57550f713896e5d9f804d0feeb"
server_key_iv: "80badf9b35cc3a83a4e0534db6709bf1"
server_key_version: "encryptionkey@0"
I20260806 00:52:50.839331 9236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "859ff4d702b947369b9fe282ed78315f"
format_stamp: "Formatted at 2026-08-06 00:52:50 on dist-test-slave-45dx"
server_key: "bfad9c57550f713896e5d9f804d0feeb"
server_key_iv: "80badf9b35cc3a83a4e0534db6709bf1"
server_key_version: "encryptionkey@0"
I20260806 00:52:50.843122 9236 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260806 00:52:50.845692 9260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:50.846859 9236 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:52:50.846998 9236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "859ff4d702b947369b9fe282ed78315f"
format_stamp: "Formatted at 2026-08-06 00:52:50 on dist-test-slave-45dx"
server_key: "bfad9c57550f713896e5d9f804d0feeb"
server_key_iv: "80badf9b35cc3a83a4e0534db6709bf1"
server_key_version: "encryptionkey@0"
I20260806 00:52:50.847110 9236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:50.862231 9236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:50.862802 9236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:50.862952 9236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:50.863466 9236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:50.864288 9236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:50.864406 9236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:50.864446 9236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:50.864462 9236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:50.873845 9236 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:46737
I20260806 00:52:50.873986 9431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:46737 every 8 connection(s)
I20260806 00:52:50.874817 9236 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:52:50.880370 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 9236
I20260806 00:52:50.880522 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:52:50.880857 15950 external_mini_cluster.cc:1428] Setting key 9587b67d7f255b12bccff3d22efad4c1
I20260806 00:52:50.883127 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39909
--builtin_ntp_servers=127.15.147.148:35903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 00:52:50.885562 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:50.876790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:57155 (local address 127.15.147.190:39909)
0806 00:52:50.876945 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:50.876949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:50.878168 (+ 1219us) server_negotiation.cc:407] Connection header received
0806 00:52:50.879823 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:50.879827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:50.879897 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:50.880014 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:50.881262 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.882008 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.883385 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.883622 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.883685 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:50.885030 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:50.885034 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:50.885037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:52:50.885050 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:50.885080 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:52:50.885088 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:50.885169 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:50.885344 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 00:52:50.885413 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 00:52:50.886590 9434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:50.877200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:39909 (local address 127.15.147.130:57155)
0806 00:52:50.877981 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:50.878029 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:50.879555 (+ 1526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:50.880210 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:50.880217 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:50.880379 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:50.881102 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:50.881119 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.882197 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.882201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:50.883166 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:50.883177 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:50.883852 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:50.883855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:50.884007 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:50.884743 (+ 736us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:52:50.884744 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:50.884790 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:52:50.884864 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:50.885183 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:50.885196 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:50.885209 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 00:52:50.885342 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 00:52:50.885665 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":133,"threads_started":1}
I20260806 00:52:50.887845 9432 heartbeater.cc:347] Connected to a master server at 127.15.147.190:39909
I20260806 00:52:50.888111 9432 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:50.888792 9432 heartbeater.cc:511] Master 127.15.147.190:39909 requested a full tablet report, sending...
I20260806 00:52:50.890004 8955 ts_manager.cc:194] Registered new tserver with Master: 859ff4d702b947369b9fe282ed78315f (127.15.147.130:46737)
I20260806 00:52:50.890969 8955 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.147.130:57155
W20260806 00:52:50.999505 9437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:50.999748 9437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:50.999782 9437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:51.003681 9437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:51.003736 9437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:51.003786 9437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:52:51.008524 9437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:35903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:39909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:51.009559 9437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:51.010874 9437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:52:51.020679 9437 server_base.cc:1034] running on GCE node
W20260806 00:52:51.020699 9443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:51.020630 9445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:51.020627 9447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:51.021273 9437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:51.021920 9437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:51.023102 9437 hybrid_clock.cc:648] HybridClock initialized: now 1785977571023070 us; error 45 us; skew 500 ppm
I20260806 00:52:51.025123 9437 webserver.cc:460] Webserver started at http://127.15.147.131:35715/ using document root <none> and password file <none>
I20260806 00:52:51.025723 9437 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:51.025789 9437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:51.026021 9437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:51.028157 9437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "b8e11223696f4187b290dec2f8354ebf"
format_stamp: "Formatted at 2026-08-06 00:52:51 on dist-test-slave-45dx"
server_key: "e16da7d87c9339ee1e167452b64faaac"
server_key_iv: "99c492ecaecc4d729d56edc4471e42c0"
server_key_version: "encryptionkey@0"
I20260806 00:52:51.028815 9437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "b8e11223696f4187b290dec2f8354ebf"
format_stamp: "Formatted at 2026-08-06 00:52:51 on dist-test-slave-45dx"
server_key: "e16da7d87c9339ee1e167452b64faaac"
server_key_iv: "99c492ecaecc4d729d56edc4471e42c0"
server_key_version: "encryptionkey@0"
I20260806 00:52:51.032668 9437 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:52:51.035188 9453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:51.036517 9437 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:52:51.036640 9437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "b8e11223696f4187b290dec2f8354ebf"
format_stamp: "Formatted at 2026-08-06 00:52:51 on dist-test-slave-45dx"
server_key: "e16da7d87c9339ee1e167452b64faaac"
server_key_iv: "99c492ecaecc4d729d56edc4471e42c0"
server_key_version: "encryptionkey@0"
I20260806 00:52:51.036764 9437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:51.050701 9437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:51.051534 9437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:51.051736 9437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:51.052457 9437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:51.053526 9437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:51.053574 9437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:51.053614 9437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:51.053637 9437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:51.062901 9437 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:37551
I20260806 00:52:51.062922 9624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:37551 every 8 connection(s)
I20260806 00:52:51.063838 9437 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:52:51.071869 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 9437
I20260806 00:52:51.071965 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:52:51.072242 15950 external_mini_cluster.cc:1428] Setting key cb478df256b913c4343c5e789c658086
I20260806 00:52:51.073380 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:51.065784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:42183 (local address 127.15.147.190:39909)
0806 00:52:51.065939 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:51.065943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:51.067067 (+ 1124us) server_negotiation.cc:407] Connection header received
0806 00:52:51.068555 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:51.068558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:51.068620 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:51.068724 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:51.069621 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.070146 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.071246 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.071460 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.071570 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:51.072796 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:51.072800 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:51.072806 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:52:51.072821 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:51.072846 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:52:51.072866 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:51.072963 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:51.073180 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 00:52:51.073232 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 00:52:51.074322 9628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:51.066191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:39909 (local address 127.15.147.131:42183)
0806 00:52:51.066905 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:51.066948 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:51.068338 (+ 1390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:51.068899 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:51.068906 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:51.068988 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:51.069471 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:51.069482 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.070277 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.070281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:51.071118 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:51.071127 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.071612 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.071616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:51.071775 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:51.072477 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:52:51.072479 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:51.072518 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:52:51.072610 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:51.072984 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:51.073001 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:51.073012 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 00:52:51.073170 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 00:52:51.073472 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":134,"threads_started":1}
I20260806 00:52:51.075417 9625 heartbeater.cc:347] Connected to a master server at 127.15.147.190:39909
I20260806 00:52:51.075663 9625 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:51.076177 9625 heartbeater.cc:511] Master 127.15.147.190:39909 requested a full tablet report, sending...
I20260806 00:52:51.077144 8955 ts_manager.cc:194] Registered new tserver with Master: b8e11223696f4187b290dec2f8354ebf (127.15.147.131:37551)
I20260806 00:52:51.077705 8955 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.147.131:42183
I20260806 00:52:51.086174 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:52:51.091928 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:51.088156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35380 (local address 127.15.147.190:39909)
0806 00:52:51.088274 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:51.088277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:51.088458 (+ 181us) server_negotiation.cc:407] Connection header received
0806 00:52:51.088598 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:51.088601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:51.088645 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:51.088721 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:51.089002 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.089705 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.090513 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.090731 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.090809 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:51.091564 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:51.091567 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:51.091569 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:52:51.091580 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:51.091599 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:52:51.091606 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:51.091663 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:51.091756 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 00:52:51.091823 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 00:52:51.094763 8955 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35380:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:52:51.097822 8955 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:52:51.112171 9650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:51.106193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:46737 (local address 127.0.0.1:33800)
0806 00:52:51.107435 (+ 1242us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:51.107445 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:51.107521 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:51.107977 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:51.107983 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:51.108003 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:51.108166 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:51.108173 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.109168 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.109174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:51.110618 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:51.110629 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.111640 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.111644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:51.111746 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:51.111782 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 00:52:51.111904 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 00:52:51.112023 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1196,"thread_start_us":111,"threads_started":1}
I20260806 00:52:51.112246 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:51.106909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33800 (local address 127.15.147.130:46737)
0806 00:52:51.107316 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:51.107325 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:51.107550 (+ 225us) server_negotiation.cc:407] Connection header received
0806 00:52:51.107636 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:51.107644 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:51.107816 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:51.107985 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:51.108297 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.109056 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.110777 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.111494 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.111628 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:51.111801 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:51.111955 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 00:52:51.112078 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":139,"threads_started":1}
I20260806 00:52:51.112572 9654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:51.106909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46696 (local address 127.15.147.129:46235)
0806 00:52:51.107361 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:51.107371 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:51.107628 (+ 257us) server_negotiation.cc:407] Connection header received
0806 00:52:51.107714 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:51.107719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:51.107903 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:51.108040 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:51.108500 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.109589 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.110844 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.111506 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.111651 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:51.111813 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:51.112261 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 00:52:51.112408 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":112,"threads_started":1}
I20260806 00:52:51.112195 9643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:51.106450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:46235 (local address 127.0.0.1:46696)
0806 00:52:51.107480 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:51.107516 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:51.107611 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:51.108207 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:51.108210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:51.108240 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:51.108356 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:51.108362 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.109720 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.109724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:51.110664 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:51.110674 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.111878 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.111883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:51.111962 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:51.111982 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:52:51.112043 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 00:52:51.112096 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":984,"mutex_wait_us":67,"spinlock_wait_cycles":768,"thread_start_us":79,"threads_started":1}
I20260806 00:52:51.113517 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:51.106238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53070 (local address 127.15.147.131:37551)
0806 00:52:51.107771 (+ 1533us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:51.107781 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:51.107800 (+ 19us) server_negotiation.cc:407] Connection header received
0806 00:52:51.107859 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:51.107863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:51.108011 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:51.108174 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:51.108464 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.109589 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.111156 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.111983 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.113068 (+ 1085us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:51.113218 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:51.113263 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 00:52:51.113354 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1439,"thread_start_us":185,"threads_started":1}
I20260806 00:52:51.113744 9642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:51.106110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:37551 (local address 127.0.0.1:53070)
0806 00:52:51.106912 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:51.106959 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:51.107071 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:51.108170 (+ 1099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:51.108172 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:51.108188 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:51.108332 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:51.108339 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.109720 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.109724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:51.111029 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:51.111040 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.112114 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.112118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:51.112207 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:51.112230 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 00:52:51.112550 (+ 320us) client_negotiation.cc:240] Negotiation successful
0806 00:52:51.112780 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":132,"spinlock_wait_cycles":22144,"thread_start_us":85,"threads_started":1}
I20260806 00:52:51.115365 9141 tablet_service.cc:1467] Processing CreateTablet for tablet a542e89b89fa457a8170eaf794b38544 (DEFAULT_TABLE table=test-table [id=f52184d966754f01b985eb2d4261f72c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:51.115492 9331 tablet_service.cc:1467] Processing CreateTablet for tablet a542e89b89fa457a8170eaf794b38544 (DEFAULT_TABLE table=test-table [id=f52184d966754f01b985eb2d4261f72c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:51.116472 9141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a542e89b89fa457a8170eaf794b38544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:51.116556 9331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a542e89b89fa457a8170eaf794b38544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:51.116158 9528 tablet_service.cc:1467] Processing CreateTablet for tablet a542e89b89fa457a8170eaf794b38544 (DEFAULT_TABLE table=test-table [id=f52184d966754f01b985eb2d4261f72c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:51.117081 9528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a542e89b89fa457a8170eaf794b38544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:51.123152 9656 tablet_bootstrap.cc:492] T a542e89b89fa457a8170eaf794b38544 P 4355a584422f47e294e54ab2dc1c4c4e: Bootstrap starting.
I20260806 00:52:51.124563 9657 tablet_bootstrap.cc:492] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf: Bootstrap starting.
I20260806 00:52:51.124588 9658 tablet_bootstrap.cc:492] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f: Bootstrap starting.
I20260806 00:52:51.125377 9656 tablet_bootstrap.cc:654] T a542e89b89fa457a8170eaf794b38544 P 4355a584422f47e294e54ab2dc1c4c4e: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:51.126186 9656 log.cc:824] T a542e89b89fa457a8170eaf794b38544 P 4355a584422f47e294e54ab2dc1c4c4e: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:51.126673 9658 tablet_bootstrap.cc:654] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:51.126673 9657 tablet_bootstrap.cc:654] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:51.127508 9657 log.cc:824] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:51.127508 9658 log.cc:824] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:51.129570 9658 tablet_bootstrap.cc:492] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f: No bootstrap required, opened a new log
I20260806 00:52:51.129609 9656 tablet_bootstrap.cc:492] T a542e89b89fa457a8170eaf794b38544 P 4355a584422f47e294e54ab2dc1c4c4e: No bootstrap required, opened a new log
I20260806 00:52:51.129760 9657 tablet_bootstrap.cc:492] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf: No bootstrap required, opened a new log
I20260806 00:52:51.129781 9656 ts_tablet_manager.cc:1397] T a542e89b89fa457a8170eaf794b38544 P 4355a584422f47e294e54ab2dc1c4c4e: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 00:52:51.129776 9658 ts_tablet_manager.cc:1397] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:52:51.129922 9657 ts_tablet_manager.cc:1397] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:52:51.133090 9658 raft_consensus.cc:348] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e11223696f4187b290dec2f8354ebf" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37551 } } peers { permanent_uuid: "859ff4d702b947369b9fe282ed78315f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46737 } } peers { permanent_uuid: "4355a584422f47e294e54ab2dc1c4c4e" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46235 } }
I20260806 00:52:51.133332 9658 raft_consensus.cc:374] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:51.133558 9658 raft_consensus.cc:729] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 859ff4d702b947369b9fe282ed78315f, State: Initialized, Role: FOLLOWER
I20260806 00:52:51.134081 9658 consensus_queue.cc:260] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e11223696f4187b290dec2f8354ebf" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37551 } } peers { permanent_uuid: "859ff4d702b947369b9fe282ed78315f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46737 } } peers { permanent_uuid: "4355a584422f47e294e54ab2dc1c4c4e" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46235 } }
I20260806 00:52:51.134099 9656 raft_consensus.cc:348] T a542e89b89fa457a8170eaf794b38544 P 4355a584422f47e294e54ab2dc1c4c4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e11223696f4187b290dec2f8354ebf" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37551 } } peers { permanent_uuid: "859ff4d702b947369b9fe282ed78315f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46737 } } peers { permanent_uuid: "4355a584422f47e294e54ab2dc1c4c4e" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46235 } }
I20260806 00:52:51.134099 9657 raft_consensus.cc:348] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e11223696f4187b290dec2f8354ebf" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37551 } } peers { permanent_uuid: "859ff4d702b947369b9fe282ed78315f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46737 } } peers { permanent_uuid: "4355a584422f47e294e54ab2dc1c4c4e" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46235 } }
I20260806 00:52:51.134363 9656 raft_consensus.cc:374] T a542e89b89fa457a8170eaf794b38544 P 4355a584422f47e294e54ab2dc1c4c4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:51.134373 9657 raft_consensus.cc:374] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:51.134419 9657 raft_consensus.cc:729] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e11223696f4187b290dec2f8354ebf, State: Initialized, Role: FOLLOWER
I20260806 00:52:51.134419 9656 raft_consensus.cc:729] T a542e89b89fa457a8170eaf794b38544 P 4355a584422f47e294e54ab2dc1c4c4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4355a584422f47e294e54ab2dc1c4c4e, State: Initialized, Role: FOLLOWER
I20260806 00:52:51.134846 9432 heartbeater.cc:503] Master 127.15.147.190:39909 was elected leader, sending a full tablet report...
I20260806 00:52:51.134881 9656 consensus_queue.cc:260] T a542e89b89fa457a8170eaf794b38544 P 4355a584422f47e294e54ab2dc1c4c4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e11223696f4187b290dec2f8354ebf" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37551 } } peers { permanent_uuid: "859ff4d702b947369b9fe282ed78315f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46737 } } peers { permanent_uuid: "4355a584422f47e294e54ab2dc1c4c4e" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46235 } }
I20260806 00:52:51.134878 9657 consensus_queue.cc:260] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e11223696f4187b290dec2f8354ebf" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37551 } } peers { permanent_uuid: "859ff4d702b947369b9fe282ed78315f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46737 } } peers { permanent_uuid: "4355a584422f47e294e54ab2dc1c4c4e" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46235 } }
I20260806 00:52:51.135129 9658 ts_tablet_manager.cc:1428] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 00:52:51.135785 9657 ts_tablet_manager.cc:1428] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:52:51.135785 9656 ts_tablet_manager.cc:1428] T a542e89b89fa457a8170eaf794b38544 P 4355a584422f47e294e54ab2dc1c4c4e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 00:52:51.136150 9625 heartbeater.cc:503] Master 127.15.147.190:39909 was elected leader, sending a full tablet report...
I20260806 00:52:51.136343 9232 heartbeater.cc:503] Master 127.15.147.190:39909 was elected leader, sending a full tablet report...
W20260806 00:52:51.182415 9234 tablet.cc:2369] T a542e89b89fa457a8170eaf794b38544 P 4355a584422f47e294e54ab2dc1c4c4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:52:51.314989 9627 tablet.cc:2369] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:52:51.376256 9433 tablet.cc:2369] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:52:51.423105 9662 raft_consensus.cc:482] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:52:51.423336 9662 raft_consensus.cc:504] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e11223696f4187b290dec2f8354ebf" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37551 } } peers { permanent_uuid: "859ff4d702b947369b9fe282ed78315f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46737 } } peers { permanent_uuid: "4355a584422f47e294e54ab2dc1c4c4e" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46235 } }
I20260806 00:52:51.424618 9662 leader_election.cc:288] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8e11223696f4187b290dec2f8354ebf (127.15.147.131:37551), 4355a584422f47e294e54ab2dc1c4c4e (127.15.147.129:46235)
I20260806 00:52:51.430117 9654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:51.425047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:40337 (local address 127.15.147.129:46235)
0806 00:52:51.425179 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:51.425185 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:51.425450 (+ 265us) server_negotiation.cc:407] Connection header received
0806 00:52:51.425588 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:51.425591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:51.425642 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:51.425737 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:51.426188 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.426976 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.428818 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.429452 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.429585 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:51.429638 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:51.429907 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 00:52:51.429980 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:52:51.430117 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:51.425047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:34111 (local address 127.15.147.131:37551)
0806 00:52:51.425179 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:51.425185 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:51.425428 (+ 243us) server_negotiation.cc:407] Connection header received
0806 00:52:51.425578 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:51.425582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:51.425640 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:51.425737 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:51.426389 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.427093 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.428810 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.429452 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.429585 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:51.429638 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:51.429907 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 00:52:51.429983 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:52:51.430207 9665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:51.424935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:37551 (local address 127.15.147.130:34111)
0806 00:52:51.425306 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:51.425323 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:51.425456 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:51.425771 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:51.425774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:51.425804 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:51.426021 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:51.426026 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.427200 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.427203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:51.428669 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:51.428677 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.429593 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.429596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:51.429752 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:51.429797 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 00:52:51.429883 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 00:52:51.429989 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":201,"spinlock_wait_cycles":38784,"thread_start_us":87,"threads_started":1}
I20260806 00:52:51.430207 9667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:51.424935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:46235 (local address 127.15.147.130:40337)
0806 00:52:51.425359 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:51.425373 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:51.425456 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:51.425771 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:51.425774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:51.425799 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:51.426016 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:51.426023 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.427094 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.427097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:51.428663 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:51.428677 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.429593 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.429596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:51.429758 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:51.429802 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 00:52:51.429877 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 00:52:51.429989 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":256,"spinlock_wait_cycles":7296,"thread_start_us":73,"threads_started":1}
I20260806 00:52:51.430943 9166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a542e89b89fa457a8170eaf794b38544" candidate_uuid: "859ff4d702b947369b9fe282ed78315f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4355a584422f47e294e54ab2dc1c4c4e" is_pre_election: true
I20260806 00:52:51.431011 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a542e89b89fa457a8170eaf794b38544" candidate_uuid: "859ff4d702b947369b9fe282ed78315f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e11223696f4187b290dec2f8354ebf" is_pre_election: true
I20260806 00:52:51.431288 9166 raft_consensus.cc:2454] T a542e89b89fa457a8170eaf794b38544 P 4355a584422f47e294e54ab2dc1c4c4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 859ff4d702b947369b9fe282ed78315f in term 0.
I20260806 00:52:51.431318 9555 raft_consensus.cc:2454] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 859ff4d702b947369b9fe282ed78315f in term 0.
I20260806 00:52:51.431856 9262 leader_election.cc:302] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4355a584422f47e294e54ab2dc1c4c4e, 859ff4d702b947369b9fe282ed78315f; no voters:
I20260806 00:52:51.432147 9662 raft_consensus.cc:2781] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:52:51.432236 9662 raft_consensus.cc:482] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:52:51.432267 9662 raft_consensus.cc:3037] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:51.433383 9662 raft_consensus.cc:504] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e11223696f4187b290dec2f8354ebf" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37551 } } peers { permanent_uuid: "859ff4d702b947369b9fe282ed78315f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46737 } } peers { permanent_uuid: "4355a584422f47e294e54ab2dc1c4c4e" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46235 } }
I20260806 00:52:51.433768 9662 leader_election.cc:288] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [CANDIDATE]: Term 1 election: Requested vote from peers b8e11223696f4187b290dec2f8354ebf (127.15.147.131:37551), 4355a584422f47e294e54ab2dc1c4c4e (127.15.147.129:46235)
I20260806 00:52:51.434139 9166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a542e89b89fa457a8170eaf794b38544" candidate_uuid: "859ff4d702b947369b9fe282ed78315f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4355a584422f47e294e54ab2dc1c4c4e"
I20260806 00:52:51.434126 9555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a542e89b89fa457a8170eaf794b38544" candidate_uuid: "859ff4d702b947369b9fe282ed78315f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8e11223696f4187b290dec2f8354ebf"
I20260806 00:52:51.434255 9555 raft_consensus.cc:3037] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:51.434254 9166 raft_consensus.cc:3037] T a542e89b89fa457a8170eaf794b38544 P 4355a584422f47e294e54ab2dc1c4c4e [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:51.435534 9166 raft_consensus.cc:2454] T a542e89b89fa457a8170eaf794b38544 P 4355a584422f47e294e54ab2dc1c4c4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 859ff4d702b947369b9fe282ed78315f in term 1.
I20260806 00:52:51.435534 9555 raft_consensus.cc:2454] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 859ff4d702b947369b9fe282ed78315f in term 1.
I20260806 00:52:51.435846 9262 leader_election.cc:302] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4355a584422f47e294e54ab2dc1c4c4e, 859ff4d702b947369b9fe282ed78315f; no voters:
I20260806 00:52:51.436030 9662 raft_consensus.cc:2781] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:52:51.436244 9662 raft_consensus.cc:686] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [term 1 LEADER]: Becoming Leader. State: Replica: 859ff4d702b947369b9fe282ed78315f, State: Running, Role: LEADER
I20260806 00:52:51.436594 9662 consensus_queue.cc:237] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e11223696f4187b290dec2f8354ebf" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37551 } } peers { permanent_uuid: "859ff4d702b947369b9fe282ed78315f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46737 } } peers { permanent_uuid: "4355a584422f47e294e54ab2dc1c4c4e" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46235 } }
I20260806 00:52:51.439702 8952 catalog_manager.cc:5495] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f reported cstate change: term changed from 0 to 1, leader changed from <none> to 859ff4d702b947369b9fe282ed78315f (127.15.147.130). New cstate: current_term: 1 leader_uuid: "859ff4d702b947369b9fe282ed78315f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8e11223696f4187b290dec2f8354ebf" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "859ff4d702b947369b9fe282ed78315f" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 46737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4355a584422f47e294e54ab2dc1c4c4e" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46235 } health_report { overall_health: UNKNOWN } } }
I20260806 00:52:51.474272 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:51.470913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33814 (local address 127.15.147.130:46737)
0806 00:52:51.471049 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:51.471052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:51.471065 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:52:51.471138 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:51.471140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:51.471196 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:51.471282 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:52:51.471495 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.471969 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.472880 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:51.473086 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:51.473164 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:51.473428 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:52:51.473873 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:52:51.473994 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:51.474071 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 00:52:51.474126 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:52:51.479943 9166 raft_consensus.cc:1261] T a542e89b89fa457a8170eaf794b38544 P 4355a584422f47e294e54ab2dc1c4c4e [term 1 FOLLOWER]: Refusing update from remote peer 859ff4d702b947369b9fe282ed78315f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:52:51.479954 9555 raft_consensus.cc:1261] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf [term 1 FOLLOWER]: Refusing update from remote peer 859ff4d702b947369b9fe282ed78315f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:52:51.480659 9668 consensus_queue.cc:1035] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4355a584422f47e294e54ab2dc1c4c4e" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:52:51.480939 9662 consensus_queue.cc:1035] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8e11223696f4187b290dec2f8354ebf" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 37551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:52:51.498298 8952 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:52:51.498497 8952 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:52:51.501180 8952 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0873a0c4176d40c789cdee0b8f5d73b1: Sending DeleteTablet for 3 replicas of tablet a542e89b89fa457a8170eaf794b38544
I20260806 00:52:51.502027 9331 tablet_service.cc:1514] Processing DeleteTablet for tablet a542e89b89fa457a8170eaf794b38544 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:51 UTC) from {username='slave'} at 127.0.0.1:33800
I20260806 00:52:51.502048 9528 tablet_service.cc:1514] Processing DeleteTablet for tablet a542e89b89fa457a8170eaf794b38544 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:51 UTC) from {username='slave'} at 127.0.0.1:53070
I20260806 00:52:51.502274 9141 tablet_service.cc:1514] Processing DeleteTablet for tablet a542e89b89fa457a8170eaf794b38544 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:51 UTC) from {username='slave'} at 127.0.0.1:46696
I20260806 00:52:51.502581 9687 tablet_replica.cc:330] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf: stopping tablet replica
I20260806 00:52:51.502969 9689 tablet_replica.cc:330] T a542e89b89fa457a8170eaf794b38544 P 4355a584422f47e294e54ab2dc1c4c4e: stopping tablet replica
I20260806 00:52:51.502961 9687 raft_consensus.cc:2229] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:52:51.503268 9689 raft_consensus.cc:2229] T a542e89b89fa457a8170eaf794b38544 P 4355a584422f47e294e54ab2dc1c4c4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:52:51.503298 9687 raft_consensus.cc:2258] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:51.503629 9689 raft_consensus.cc:2258] T a542e89b89fa457a8170eaf794b38544 P 4355a584422f47e294e54ab2dc1c4c4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:51.503710 9688 tablet_replica.cc:330] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f: stopping tablet replica
I20260806 00:52:51.503811 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 9040
I20260806 00:52:51.503991 9688 raft_consensus.cc:2229] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:52:51.504417 9688 raft_consensus.cc:2258] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:51.505798 9687 ts_tablet_manager.cc:1905] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:52:51.508796 9687 ts_tablet_manager.cc:1918] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:52:51.508903 9687 log.cc:1196] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/a542e89b89fa457a8170eaf794b38544
I20260806 00:52:51.509207 9687 ts_tablet_manager.cc:1939] T a542e89b89fa457a8170eaf794b38544 P b8e11223696f4187b290dec2f8354ebf: Deleting consensus metadata
I20260806 00:52:51.510473 8933 catalog_manager.cc:4841] TS b8e11223696f4187b290dec2f8354ebf (127.15.147.131:37551): tablet a542e89b89fa457a8170eaf794b38544 (table test-table [id=f52184d966754f01b985eb2d4261f72c]) successfully deleted
I20260806 00:52:51.511479 9688 ts_tablet_manager.cc:1905] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:52:51.514067 9688 ts_tablet_manager.cc:1918] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:52:51.514183 9688 log.cc:1196] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/a542e89b89fa457a8170eaf794b38544
I20260806 00:52:51.514508 9688 ts_tablet_manager.cc:1939] T a542e89b89fa457a8170eaf794b38544 P 859ff4d702b947369b9fe282ed78315f: Deleting consensus metadata
I20260806 00:52:51.515563 8931 catalog_manager.cc:4841] TS 859ff4d702b947369b9fe282ed78315f (127.15.147.130:46737): tablet a542e89b89fa457a8170eaf794b38544 (table test-table [id=f52184d966754f01b985eb2d4261f72c]) successfully deleted
W20260806 00:52:51.520102 8932 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.147.129:46235 (error 108)
I20260806 00:52:51.520357 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 9236
I20260806 00:52:51.523442 9642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:51.523156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:46235 (local address 127.0.0.1:46706)
0806 00:52:51.523291 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:51.523367 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:46235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 00:52:51.523721 8932 catalog_manager.cc:4584] TS 4355a584422f47e294e54ab2dc1c4c4e (127.15.147.129:46235): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a542e89b89fa457a8170eaf794b38544: Network error: Client connection negotiation failed: client connection to 127.15.147.129:46235: connect: Connection refused (error 111)
I20260806 00:52:51.528489 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 9437
I20260806 00:52:51.542763 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 8915
2026-08-06T00:52:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1378 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:52:51 dist-test-slave-45dx krb5kdc[9693](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:52:51 dist-test-slave-45dx krb5kdc[9693](info): set up 2 sockets
Aug 06 00:52:51 dist-test-slave-45dx krb5kdc[9693](info): commencing operation
krb5kdc: starting...
W20260806 00:52:52.757135 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:52:52 dist-test-slave-45dx krb5kdc[9693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:52:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:52:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:52:52.891870 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:36063
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:45087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:36063
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:52:53.008236 9712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:53.008682 9712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:53.008766 9712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:53.012693 9712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:52:53.012743 9712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:53.012761 9712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:53.012778 9712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:52:53.012794 9712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:52:53.017926 9712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:45087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:36063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:36063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:53.018988 9712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:53.019972 9712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:53.028672 9718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:53.028659 9712 server_base.cc:1034] running on GCE node
W20260806 00:52:53.028664 9719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:53.028708 9722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:53.029510 9712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:53.030696 9712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:53.031899 9712 hybrid_clock.cc:648] HybridClock initialized: now 1785977573031873 us; error 44 us; skew 500 ppm
Aug 06 00:52:53 dist-test-slave-45dx krb5kdc[9693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977573, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:53.045559 9712 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:52:53.046612 9712 webserver.cc:460] Webserver started at http://127.15.147.190:36269/ using document root <none> and password file <none>
I20260806 00:52:53.047223 9712 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:53.047292 9712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:53.047478 9712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:53.049925 9712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "8f8f73ef50f945c0b5164d97dad3e584"
format_stamp: "Formatted at 2026-08-06 00:52:53 on dist-test-slave-45dx"
server_key: "3c7f36cd5b3dd3191d51d320fcc506c6"
server_key_iv: "9fd9f1df64aa015f4a26058bb871212f"
server_key_version: "encryptionkey@0"
I20260806 00:52:53.050518 9712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "8f8f73ef50f945c0b5164d97dad3e584"
format_stamp: "Formatted at 2026-08-06 00:52:53 on dist-test-slave-45dx"
server_key: "3c7f36cd5b3dd3191d51d320fcc506c6"
server_key_iv: "9fd9f1df64aa015f4a26058bb871212f"
server_key_version: "encryptionkey@0"
I20260806 00:52:53.054621 9712 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:52:53.057161 9733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:53.058493 9712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:52:53.058665 9712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "8f8f73ef50f945c0b5164d97dad3e584"
format_stamp: "Formatted at 2026-08-06 00:52:53 on dist-test-slave-45dx"
server_key: "3c7f36cd5b3dd3191d51d320fcc506c6"
server_key_iv: "9fd9f1df64aa015f4a26058bb871212f"
server_key_version: "encryptionkey@0"
I20260806 00:52:53.058826 9712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:53.073479 9712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:53.076651 9712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:53.076893 9712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:53.084882 9712 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:36063
I20260806 00:52:53.084928 9810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:36063 every 8 connection(s)
I20260806 00:52:53.085970 9712 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:52:53.088992 9811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:53.089588 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 9712
I20260806 00:52:53.089735 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:52:53.090066 15950 external_mini_cluster.cc:1428] Setting key 16551ce77117f933377bf90ad6ef2cec
I20260806 00:52:53.096294 9811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584: Bootstrap starting.
I20260806 00:52:53.099171 9811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584: Neither blocks nor log segments found. Creating new log.
Aug 06 00:52:53 dist-test-slave-45dx krb5kdc[9693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:52:53.099942 9811 log.cc:824] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:53.101951 9811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584: No bootstrap required, opened a new log
I20260806 00:52:53.104775 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:53.091501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44670 (local address 127.15.147.190:36063)
0806 00:52:53.091939 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:53.091955 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:53.091991 (+ 36us) server_negotiation.cc:407] Connection header received
0806 00:52:53.092711 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:53.092730 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:53.093059 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:53.093328 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:53.094286 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.095252 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.096450 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.096971 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.097098 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:53.099878 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:53.099884 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:53.099898 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:53.099925 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:53.101869 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:53.102393 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:53.102399 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:53.102412 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:53.102526 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:53.102816 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:53.102820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:53.102822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:53.103209 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:53.103407 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:53.103710 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 00:52:53.104003 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":139,"threads_started":1}
I20260806 00:52:53.105593 9811 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f8f73ef50f945c0b5164d97dad3e584" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 36063 } }
I20260806 00:52:53.105834 9811 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:53.105927 9811 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f8f73ef50f945c0b5164d97dad3e584, State: Initialized, Role: FOLLOWER
I20260806 00:52:53.106348 9811 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f8f73ef50f945c0b5164d97dad3e584" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 36063 } }
I20260806 00:52:53.106511 9811 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:52:53.106598 9811 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:52:53.106719 9811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:53.107728 9811 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f8f73ef50f945c0b5164d97dad3e584" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 36063 } }
I20260806 00:52:53.108006 9811 leader_election.cc:302] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f8f73ef50f945c0b5164d97dad3e584; no voters:
I20260806 00:52:53.108246 9811 leader_election.cc:288] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:52:53.108392 9816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:52:53.108626 9816 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584 [term 1 LEADER]: Becoming Leader. State: Replica: 8f8f73ef50f945c0b5164d97dad3e584, State: Running, Role: LEADER
I20260806 00:52:53.108927 9816 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f8f73ef50f945c0b5164d97dad3e584" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 36063 } }
I20260806 00:52:53.109560 9811 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:52:53.110113 9818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f8f73ef50f945c0b5164d97dad3e584. Latest consensus state: current_term: 1 leader_uuid: "8f8f73ef50f945c0b5164d97dad3e584" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f8f73ef50f945c0b5164d97dad3e584" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 36063 } } }
I20260806 00:52:53.110248 9818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584 [sys.catalog]: This master's current role is: LEADER
I20260806 00:52:53.110518 9817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f8f73ef50f945c0b5164d97dad3e584" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f8f73ef50f945c0b5164d97dad3e584" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 36063 } } }
I20260806 00:52:53.110639 9817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584 [sys.catalog]: This master's current role is: LEADER
I20260806 00:52:53.110670 9821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:52:53.113812 9821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:52:53.119124 9821 catalog_manager.cc:1347] Generated new cluster ID: f42d9e89f9f64d1dbdde83df84084a4a
I20260806 00:52:53.119212 9821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:52:53.131498 9821 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:52:53.132467 9821 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:52:53.143420 9821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584: Generated new TSK 0
I20260806 00:52:53.144161 9821 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
I20260806 00:52:53.180197 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:36063
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:45087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:52:53.332352 9838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:53.332629 9838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:53.332713 9838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:53.336758 9838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:53.336818 9838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:53.336875 9838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:52:53.342224 9838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:45087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:36063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:53.343353 9838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:53.344419 9838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:53.354434 9846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:53.354420 9844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:53.354485 9838 server_base.cc:1034] running on GCE node
W20260806 00:52:53.354420 9849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:53.355105 9838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:53.355707 9838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:53.356877 9838 hybrid_clock.cc:648] HybridClock initialized: now 1785977573356859 us; error 31 us; skew 500 ppm
Aug 06 00:52:53 dist-test-slave-45dx krb5kdc[9693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977573, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:53.359810 9838 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:52:53.360680 9838 webserver.cc:460] Webserver started at http://127.15.147.129:41251/ using document root <none> and password file <none>
I20260806 00:52:53.361210 9838 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:53.361269 9838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:53.361449 9838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:53.363490 9838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "809c1352b75346bdac21294705ea8c3a"
format_stamp: "Formatted at 2026-08-06 00:52:53 on dist-test-slave-45dx"
server_key: "5e1ada96e7d9c94726a29e4887b90e79"
server_key_iv: "a99ae5f6e86f3e77faa809c6b276e035"
server_key_version: "encryptionkey@0"
I20260806 00:52:53.364023 9838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "809c1352b75346bdac21294705ea8c3a"
format_stamp: "Formatted at 2026-08-06 00:52:53 on dist-test-slave-45dx"
server_key: "5e1ada96e7d9c94726a29e4887b90e79"
server_key_iv: "a99ae5f6e86f3e77faa809c6b276e035"
server_key_version: "encryptionkey@0"
I20260806 00:52:53.367692 9838 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:52:53.370199 9858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:53.371450 9838 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:52:53.371591 9838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "809c1352b75346bdac21294705ea8c3a"
format_stamp: "Formatted at 2026-08-06 00:52:53 on dist-test-slave-45dx"
server_key: "5e1ada96e7d9c94726a29e4887b90e79"
server_key_iv: "a99ae5f6e86f3e77faa809c6b276e035"
server_key_version: "encryptionkey@0"
I20260806 00:52:53.371703 9838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:53.396462 9838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:53.400204 9838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:53.400472 9838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:53.401140 9838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:53.402061 9838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:53.402135 9838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:53.402210 9838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:53.402251 9838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:53.411659 9838 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:34079
I20260806 00:52:53.411706 10012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:34079 every 8 connection(s)
I20260806 00:52:53.412588 9838 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:52:53.419672 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 9838
I20260806 00:52:53.419838 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:52:53.420156 15950 external_mini_cluster.cc:1428] Setting key 7430f0bccdf3e36d0c88b462ad932453
Aug 06 00:52:53 dist-test-slave-45dx krb5kdc[9693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977573, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:52:53.426764 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:53.414550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:58455 (local address 127.15.147.190:36063)
0806 00:52:53.414711 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:53.414715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:53.415564 (+ 849us) server_negotiation.cc:407] Connection header received
0806 00:52:53.416986 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:53.416990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:53.417038 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:53.417118 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:53.418241 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.418796 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.419952 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.420186 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.420313 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:53.423116 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:53.423122 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:53.423129 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:53.423152 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:53.424729 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:53.425260 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:53.425264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:53.425266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:53.425357 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:53.425707 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:53.425712 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:53.425714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:53.425897 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:53.426004 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:53.426474 (+ 470us) server_negotiation.cc:299] Negotiation successful
0806 00:52:53.426596 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:52:53.427711 10018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:53.414933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:36063 (local address 127.15.147.129:58455)
0806 00:52:53.415403 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:53.415447 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:53.416767 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:53.417278 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:53.417285 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:53.417630 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:53.418094 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:53.418106 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.418904 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.418906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:53.419818 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:53.419829 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.420417 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.420424 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:53.420597 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:53.421186 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:52:53.421204 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:53.422956 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:53.424875 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:53.424881 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:53.424894 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:53.425156 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:53.425453 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:53.425457 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:53.425459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:53.425611 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:53.426003 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:53.426009 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:53.426248 (+ 239us) client_negotiation.cc:769] Sending connection context
0806 00:52:53.426431 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 00:52:53.426743 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":114,"threads_started":1}
I20260806 00:52:53.429006 10013 heartbeater.cc:347] Connected to a master server at 127.15.147.190:36063
I20260806 00:52:53.429293 10013 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:53.429916 10013 heartbeater.cc:511] Master 127.15.147.190:36063 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.147.130@KRBTEST.COM; defaulting to no policy
I20260806 00:52:53.431620 9756 ts_manager.cc:194] Registered new tserver with Master: 809c1352b75346bdac21294705ea8c3a (127.15.147.129:34079)
I20260806 00:52:53.432883 9756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:58455
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
I20260806 00:52:53.450919 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:36063
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:45087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:52:53.566583 10022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:53.566864 10022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:53.566946 10022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:53.571038 10022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:53.571092 10022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:53.571148 10022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:52:53.576447 10022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:45087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:36063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:53.577524 10022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:53.578534 10022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:53.588480 10032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:53.588480 10031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:53.588605 10022 server_base.cc:1034] running on GCE node
W20260806 00:52:53.588493 10035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:53.589162 10022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:53.589751 10022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:53.590932 10022 hybrid_clock.cc:648] HybridClock initialized: now 1785977573590916 us; error 25 us; skew 500 ppm
Aug 06 00:52:53 dist-test-slave-45dx krb5kdc[9693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977573, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:53.593802 10022 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:52:53.594704 10022 webserver.cc:460] Webserver started at http://127.15.147.130:34609/ using document root <none> and password file <none>
I20260806 00:52:53.595239 10022 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:53.595299 10022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:53.595469 10022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:53.597675 10022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "c03d6a0300ed4445b2217a8dab76b56e"
format_stamp: "Formatted at 2026-08-06 00:52:53 on dist-test-slave-45dx"
server_key: "06f3b4530a399d4ce1d5da4e182aa28b"
server_key_iv: "8ee2f8a0ac85879a9e28cd0a57e6d50d"
server_key_version: "encryptionkey@0"
I20260806 00:52:53.598225 10022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "c03d6a0300ed4445b2217a8dab76b56e"
format_stamp: "Formatted at 2026-08-06 00:52:53 on dist-test-slave-45dx"
server_key: "06f3b4530a399d4ce1d5da4e182aa28b"
server_key_iv: "8ee2f8a0ac85879a9e28cd0a57e6d50d"
server_key_version: "encryptionkey@0"
I20260806 00:52:53.602084 10022 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260806 00:52:53.604681 10046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:53.605935 10022 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:52:53.606066 10022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "c03d6a0300ed4445b2217a8dab76b56e"
format_stamp: "Formatted at 2026-08-06 00:52:53 on dist-test-slave-45dx"
server_key: "06f3b4530a399d4ce1d5da4e182aa28b"
server_key_iv: "8ee2f8a0ac85879a9e28cd0a57e6d50d"
server_key_version: "encryptionkey@0"
I20260806 00:52:53.606184 10022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:53.618445 10022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:53.621534 10022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:53.621747 10022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:53.622375 10022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:53.623215 10022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:53.623261 10022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:53.623327 10022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:53.623359 10022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:53.632968 10022 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:33897
I20260806 00:52:53.633005 10217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:33897 every 8 connection(s)
I20260806 00:52:53.633831 10022 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:52:53.638178 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 10022
I20260806 00:52:53.638327 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:52:53.638628 15950 external_mini_cluster.cc:1428] Setting key 2cd99e792013b766cbfff064320088a1
Aug 06 00:52:53 dist-test-slave-45dx krb5kdc[9693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977573, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:52:53.647814 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:53.635591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:59763 (local address 127.15.147.190:36063)
0806 00:52:53.635730 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:53.635734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:53.636569 (+ 835us) server_negotiation.cc:407] Connection header received
0806 00:52:53.637959 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:53.637962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:53.638020 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:53.638128 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:53.639459 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.640072 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.641287 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.641513 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.641626 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:53.644292 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:53.644296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:53.644312 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:53.644356 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:53.645807 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:53.646318 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:53.646322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:53.646324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:53.646410 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:53.646742 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:53.646747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:53.646749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:53.646936 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:53.647054 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:53.647531 (+ 477us) server_negotiation.cc:299] Negotiation successful
0806 00:52:53.647646 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:52:53.648708 10223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:53.635939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:36063 (local address 127.15.147.130:59763)
0806 00:52:53.636427 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:53.636469 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:53.637750 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:53.638306 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:53.638313 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:53.638751 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:53.639294 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:53.639315 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.640195 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.640199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:53.641154 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:53.641165 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.641666 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.641671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:53.641851 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:53.642465 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:52:53.642482 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:53.644126 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:53.645937 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:53.645942 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:53.645956 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:53.646216 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:53.646501 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:53.646506 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:53.646508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:53.646647 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:53.647044 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:53.647051 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:53.647293 (+ 242us) client_negotiation.cc:769] Sending connection context
0806 00:52:53.647483 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 00:52:53.647791 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":122,"threads_started":1}
WARNING: no policy specified for kudu/127.15.147.131@KRBTEST.COM; defaulting to no policy
I20260806 00:52:53.649876 10221 heartbeater.cc:347] Connected to a master server at 127.15.147.190:36063
I20260806 00:52:53.650154 10221 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:53.650699 10221 heartbeater.cc:511] Master 127.15.147.190:36063 requested a full tablet report, sending...
I20260806 00:52:53.651793 9756 ts_manager.cc:194] Registered new tserver with Master: c03d6a0300ed4445b2217a8dab76b56e (127.15.147.130:33897)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.131@KRBTEST.COM" created.
I20260806 00:52:53.652637 9756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:59763
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
I20260806 00:52:53.667326 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:36063
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:45087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:52:53.782090 10226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:53.782346 10226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:53.782389 10226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:53.786255 10226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:53.786303 10226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:53.786360 10226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:52:53.791461 10226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:45087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:36063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:53.792450 10226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:53.793408 10226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:52:53.803185 10226 server_base.cc:1034] running on GCE node
W20260806 00:52:53.803179 10232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:53.803232 10234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:53.803213 10237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:53.803892 10226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:53.804664 10226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:53.805872 10226 hybrid_clock.cc:648] HybridClock initialized: now 1785977573805859 us; error 28 us; skew 500 ppm
Aug 06 00:52:53 dist-test-slave-45dx krb5kdc[9693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977573, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:53.808830 10226 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:52:53.809805 10226 webserver.cc:460] Webserver started at http://127.15.147.131:42805/ using document root <none> and password file <none>
I20260806 00:52:53.810468 10226 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:53.810556 10226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:53.810886 10226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:53.813414 10226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "ca1a58e190114c7198a21d6f2a1b9354"
format_stamp: "Formatted at 2026-08-06 00:52:53 on dist-test-slave-45dx"
server_key: "c40acf2cb9a25316ac7edf9d92d53638"
server_key_iv: "6241da41c7975d8fb31ccf768d1fee3e"
server_key_version: "encryptionkey@0"
I20260806 00:52:53.813951 10226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "ca1a58e190114c7198a21d6f2a1b9354"
format_stamp: "Formatted at 2026-08-06 00:52:53 on dist-test-slave-45dx"
server_key: "c40acf2cb9a25316ac7edf9d92d53638"
server_key_iv: "6241da41c7975d8fb31ccf768d1fee3e"
server_key_version: "encryptionkey@0"
I20260806 00:52:53.817780 10226 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:52:53.820353 10246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:53.821673 10226 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:52:53.821790 10226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "ca1a58e190114c7198a21d6f2a1b9354"
format_stamp: "Formatted at 2026-08-06 00:52:53 on dist-test-slave-45dx"
server_key: "c40acf2cb9a25316ac7edf9d92d53638"
server_key_iv: "6241da41c7975d8fb31ccf768d1fee3e"
server_key_version: "encryptionkey@0"
I20260806 00:52:53.821899 10226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:53.843729 10226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:53.846792 10226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:53.846958 10226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:53.847499 10226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:53.848404 10226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:53.848454 10226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:53.848495 10226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:53.848515 10226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:53.857501 10226 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:38591
I20260806 00:52:53.857542 10412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:38591 every 8 connection(s)
I20260806 00:52:53.858394 10226 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:52:53.864588 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 10226
I20260806 00:52:53.864745 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:52:53.865110 15950 external_mini_cluster.cc:1428] Setting key ee20e5069388793c8654f5b7b8ff1c12
Aug 06 00:52:53 dist-test-slave-45dx krb5kdc[9693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977573, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:52:53.872671 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:53.860171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:50435 (local address 127.15.147.190:36063)
0806 00:52:53.860357 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:53.860363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:53.861218 (+ 855us) server_negotiation.cc:407] Connection header received
0806 00:52:53.862635 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:53.862638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:53.862696 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:53.862772 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:53.864047 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.864588 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.866211 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.866540 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.866642 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:53.869216 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:53.869219 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:53.869221 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:53.869245 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:53.870646 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:53.871156 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:53.871161 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:53.871163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:53.871248 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:53.871597 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:53.871601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:53.871603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:53.871781 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:53.871890 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:53.872388 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 00:52:53.872509 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 00:52:53.873430 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:53.860620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:36063 (local address 127.15.147.131:50435)
0806 00:52:53.861063 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:53.861108 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:53.862383 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:53.862967 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:53.862973 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:53.863309 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:53.863912 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:53.863929 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.864771 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.864777 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:53.866095 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:53.866106 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.866683 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.866686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:53.866826 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:53.867424 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:52:53.867440 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:53.869068 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:53.870773 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:53.870778 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:53.870796 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:53.871051 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:53.871355 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:53.871359 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:53.871361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:53.871500 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:53.871892 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:53.871909 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:53.872141 (+ 232us) client_negotiation.cc:769] Sending connection context
0806 00:52:53.872343 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 00:52:53.872658 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":115,"threads_started":1}
I20260806 00:52:53.874509 10413 heartbeater.cc:347] Connected to a master server at 127.15.147.190:36063
I20260806 00:52:53.874750 10413 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:53.875262 10413 heartbeater.cc:511] Master 127.15.147.190:36063 requested a full tablet report, sending...
I20260806 00:52:53.876255 9756 ts_manager.cc:194] Registered new tserver with Master: ca1a58e190114c7198a21d6f2a1b9354 (127.15.147.131:38591)
I20260806 00:52:53.876860 9756 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:50435
I20260806 00:52:53.879254 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:52:53.889192 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:53.881317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44672 (local address 127.15.147.190:36063)
0806 00:52:53.881439 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:53.881455 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:53.881669 (+ 214us) server_negotiation.cc:407] Connection header received
0806 00:52:53.881811 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:53.881814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:53.881856 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:53.881947 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:53.882672 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.883185 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.884044 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.884272 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.884425 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:53.886013 (+ 1588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:53.886019 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:53.886022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:53.886047 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:53.887219 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:53.887749 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:53.887753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:53.887754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:53.887843 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:53.888178 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:53.888181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:53.888183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:53.888485 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:53.888645 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:53.888924 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 00:52:53.889020 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 00:52:53.892254 9756 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:52:53.895416 9756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:52:53.911157 10428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:53.904715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:33897 (local address 127.0.0.1:36580)
0806 00:52:53.905391 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:53.905445 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:53.905575 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:53.906616 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:53.906623 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:53.906644 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:53.906828 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:53.906836 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.907953 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.907958 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:53.909222 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:53.909232 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.910527 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.910531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:53.910651 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:53.910718 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 00:52:53.910850 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 00:52:53.910966 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":3,"thread_start_us":82,"threads_started":1}
I20260806 00:52:53.911908 10434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:53.904915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:38591 (local address 127.0.0.1:48992)
0806 00:52:53.905823 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:53.905836 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:53.905924 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:53.906686 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:53.906690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:53.906709 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:53.906860 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:53.906868 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.908520 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.908523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:53.909500 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:53.909510 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.911477 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.911487 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:53.911592 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:53.911619 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 00:52:53.911698 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 00:52:53.911771 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"mutex_wait_us":36,"thread_start_us":124,"threads_started":1}
I20260806 00:52:53.911974 10435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:53.905482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48992 (local address 127.15.147.131:38591)
0806 00:52:53.906170 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:53.906181 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:53.906205 (+ 24us) server_negotiation.cc:407] Connection header received
0806 00:52:53.906284 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:53.906291 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:53.906519 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:53.907007 (+ 488us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:53.907130 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.908348 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.909638 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.910382 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.910544 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:53.910692 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:53.911734 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0806 00:52:53.911835 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":180,"threads_started":1}
I20260806 00:52:53.912104 10431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:53.905513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:34079 (local address 127.0.0.1:42638)
0806 00:52:53.906272 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:53.906286 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:53.906394 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:53.906980 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:53.906983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:53.907003 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:53.907098 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:53.907104 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.908538 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.908541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:53.910406 (+ 1865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:53.910418 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.911695 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.911698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:53.911809 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:53.911834 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 00:52:53.911905 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 00:52:53.911978 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":97,"threads_started":1}
I20260806 00:52:53.911620 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:53.904920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36580 (local address 127.15.147.130:33897)
0806 00:52:53.906067 (+ 1147us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:53.906089 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:53.906115 (+ 26us) server_negotiation.cc:407] Connection header received
0806 00:52:53.906198 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:53.906205 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:53.906442 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:53.906629 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:53.906953 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.907809 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.909347 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.910388 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.910555 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:53.910944 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:53.910998 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 00:52:53.911094 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1029,"thread_start_us":126,"threads_started":1}
I20260806 00:52:53.912635 10436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:53.905930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42638 (local address 127.15.147.129:34079)
0806 00:52:53.906265 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:53.906280 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:53.906354 (+ 74us) server_negotiation.cc:407] Connection header received
0806 00:52:53.906519 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:53.906524 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:53.906693 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:53.906830 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:53.907221 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.908403 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.910548 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:53.911390 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:53.911503 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:53.911624 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:53.912401 (+ 777us) server_negotiation.cc:299] Negotiation successful
0806 00:52:53.912503 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":76,"threads_started":1}
I20260806 00:52:53.915138 9926 tablet_service.cc:1467] Processing CreateTablet for tablet af973c8fe86545b1988575448a8eb71e (DEFAULT_TABLE table=test-table [id=da329a5f480b49a199466b32a519bc04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:53.915160 10315 tablet_service.cc:1467] Processing CreateTablet for tablet af973c8fe86545b1988575448a8eb71e (DEFAULT_TABLE table=test-table [id=da329a5f480b49a199466b32a519bc04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:53.915159 10117 tablet_service.cc:1467] Processing CreateTablet for tablet af973c8fe86545b1988575448a8eb71e (DEFAULT_TABLE table=test-table [id=da329a5f480b49a199466b32a519bc04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:53.916242 10117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af973c8fe86545b1988575448a8eb71e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:53.916242 10315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af973c8fe86545b1988575448a8eb71e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:53.916249 9926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af973c8fe86545b1988575448a8eb71e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:53.922646 10437 tablet_bootstrap.cc:492] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a: Bootstrap starting.
I20260806 00:52:53.924163 10438 tablet_bootstrap.cc:492] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e: Bootstrap starting.
I20260806 00:52:53.924223 10439 tablet_bootstrap.cc:492] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354: Bootstrap starting.
I20260806 00:52:53.925177 10437 tablet_bootstrap.cc:654] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:53.926323 10439 tablet_bootstrap.cc:654] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:53.926445 10438 tablet_bootstrap.cc:654] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:53.927294 10437 log.cc:824] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:53.927701 10439 log.cc:824] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:53.927749 10438 log.cc:824] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:53.929587 10437 tablet_bootstrap.cc:492] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a: No bootstrap required, opened a new log
I20260806 00:52:53.929841 10437 ts_tablet_manager.cc:1397] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 00:52:53.929983 10439 tablet_bootstrap.cc:492] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354: No bootstrap required, opened a new log
I20260806 00:52:53.929978 10438 tablet_bootstrap.cc:492] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e: No bootstrap required, opened a new log
I20260806 00:52:53.930186 10439 ts_tablet_manager.cc:1397] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:52:53.930186 10438 ts_tablet_manager.cc:1397] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 00:52:53.933162 10437 raft_consensus.cc:348] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c03d6a0300ed4445b2217a8dab76b56e" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33897 } } peers { permanent_uuid: "ca1a58e190114c7198a21d6f2a1b9354" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38591 } } peers { permanent_uuid: "809c1352b75346bdac21294705ea8c3a" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 34079 } }
I20260806 00:52:53.933643 10437 raft_consensus.cc:374] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:53.933768 10437 raft_consensus.cc:729] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 809c1352b75346bdac21294705ea8c3a, State: Initialized, Role: FOLLOWER
I20260806 00:52:53.934419 10437 consensus_queue.cc:260] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c03d6a0300ed4445b2217a8dab76b56e" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33897 } } peers { permanent_uuid: "ca1a58e190114c7198a21d6f2a1b9354" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38591 } } peers { permanent_uuid: "809c1352b75346bdac21294705ea8c3a" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 34079 } }
I20260806 00:52:53.934410 10439 raft_consensus.cc:348] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c03d6a0300ed4445b2217a8dab76b56e" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33897 } } peers { permanent_uuid: "ca1a58e190114c7198a21d6f2a1b9354" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38591 } } peers { permanent_uuid: "809c1352b75346bdac21294705ea8c3a" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 34079 } }
I20260806 00:52:53.934410 10438 raft_consensus.cc:348] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c03d6a0300ed4445b2217a8dab76b56e" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33897 } } peers { permanent_uuid: "ca1a58e190114c7198a21d6f2a1b9354" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38591 } } peers { permanent_uuid: "809c1352b75346bdac21294705ea8c3a" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 34079 } }
I20260806 00:52:53.934705 10438 raft_consensus.cc:374] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:53.934705 10439 raft_consensus.cc:374] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:53.934769 10438 raft_consensus.cc:729] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c03d6a0300ed4445b2217a8dab76b56e, State: Initialized, Role: FOLLOWER
I20260806 00:52:53.934782 10439 raft_consensus.cc:729] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca1a58e190114c7198a21d6f2a1b9354, State: Initialized, Role: FOLLOWER
I20260806 00:52:53.935276 10013 heartbeater.cc:503] Master 127.15.147.190:36063 was elected leader, sending a full tablet report...
I20260806 00:52:53.935267 10439 consensus_queue.cc:260] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c03d6a0300ed4445b2217a8dab76b56e" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33897 } } peers { permanent_uuid: "ca1a58e190114c7198a21d6f2a1b9354" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38591 } } peers { permanent_uuid: "809c1352b75346bdac21294705ea8c3a" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 34079 } }
I20260806 00:52:53.935711 10437 ts_tablet_manager.cc:1428] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:52:53.936048 10413 heartbeater.cc:503] Master 127.15.147.190:36063 was elected leader, sending a full tablet report...
I20260806 00:52:53.936506 10438 consensus_queue.cc:260] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c03d6a0300ed4445b2217a8dab76b56e" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33897 } } peers { permanent_uuid: "ca1a58e190114c7198a21d6f2a1b9354" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38591 } } peers { permanent_uuid: "809c1352b75346bdac21294705ea8c3a" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 34079 } }
I20260806 00:52:53.937310 10221 heartbeater.cc:503] Master 127.15.147.190:36063 was elected leader, sending a full tablet report...
I20260806 00:52:53.937649 10438 ts_tablet_manager.cc:1428] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 00:52:53.939553 10439 ts_tablet_manager.cc:1428] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260806 00:52:54.046972 10445 raft_consensus.cc:482] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:52:54.047195 10445 raft_consensus.cc:504] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c03d6a0300ed4445b2217a8dab76b56e" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33897 } } peers { permanent_uuid: "ca1a58e190114c7198a21d6f2a1b9354" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38591 } } peers { permanent_uuid: "809c1352b75346bdac21294705ea8c3a" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 34079 } }
I20260806 00:52:54.048488 10445 leader_election.cc:288] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c03d6a0300ed4445b2217a8dab76b56e (127.15.147.130:33897), ca1a58e190114c7198a21d6f2a1b9354 (127.15.147.131:38591)
I20260806 00:52:54.052729 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:54.048858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:53165 (local address 127.15.147.130:33897)
0806 00:52:54.049002 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:54.049007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:54.049296 (+ 289us) server_negotiation.cc:407] Connection header received
0806 00:52:54.049407 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:54.049413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:54.049469 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:54.049551 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:54.049895 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.050457 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.051630 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.052028 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.052127 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:54.052212 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:54.052501 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 00:52:54.052577 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 00:52:54.052731 10435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:54.048949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:52225 (local address 127.15.147.131:38591)
0806 00:52:54.049166 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:54.049173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:54.049435 (+ 262us) server_negotiation.cc:407] Connection header received
0806 00:52:54.049577 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:54.049580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:54.049634 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:54.049715 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:54.050020 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.050541 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.051635 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.052057 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.052207 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:54.052263 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:54.052501 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 00:52:54.052579 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 00:52:54.052798 10449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:54.048726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:33897 (local address 127.15.147.129:53165)
0806 00:52:54.049177 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:54.049191 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:54.049312 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:54.049578 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:54.049581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:54.049599 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:54.049767 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:54.049775 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.050582 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.050588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:54.051491 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:54.051502 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.052180 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.052184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:54.052279 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:54.052388 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 00:52:54.052481 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 00:52:54.052580 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":23,"thread_start_us":167,"threads_started":1}
I20260806 00:52:54.052798 10451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:54.048801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:38591 (local address 127.15.147.129:52225)
0806 00:52:54.049309 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:54.049325 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:54.049444 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:54.049747 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:54.049749 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:54.049764 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:54.049901 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:54.049908 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.050658 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.050661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:54.051495 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:54.051504 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.052221 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.052227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:54.052375 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:54.052393 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 00:52:54.052483 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 00:52:54.052581 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":57,"thread_start_us":129,"threads_started":1}
I20260806 00:52:54.053607 10150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af973c8fe86545b1988575448a8eb71e" candidate_uuid: "809c1352b75346bdac21294705ea8c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c03d6a0300ed4445b2217a8dab76b56e" is_pre_election: true
I20260806 00:52:54.053629 10341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af973c8fe86545b1988575448a8eb71e" candidate_uuid: "809c1352b75346bdac21294705ea8c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1a58e190114c7198a21d6f2a1b9354" is_pre_election: true
I20260806 00:52:54.053951 10150 raft_consensus.cc:2454] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 809c1352b75346bdac21294705ea8c3a in term 0.
I20260806 00:52:54.053982 10341 raft_consensus.cc:2454] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 809c1352b75346bdac21294705ea8c3a in term 0.
I20260806 00:52:54.054539 9861 leader_election.cc:302] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 809c1352b75346bdac21294705ea8c3a, c03d6a0300ed4445b2217a8dab76b56e, ca1a58e190114c7198a21d6f2a1b9354; no voters:
I20260806 00:52:54.054880 10445 raft_consensus.cc:2781] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:52:54.054975 10445 raft_consensus.cc:482] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:52:54.055027 10445 raft_consensus.cc:3037] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:54.056507 10445 raft_consensus.cc:504] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c03d6a0300ed4445b2217a8dab76b56e" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33897 } } peers { permanent_uuid: "ca1a58e190114c7198a21d6f2a1b9354" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38591 } } peers { permanent_uuid: "809c1352b75346bdac21294705ea8c3a" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 34079 } }
I20260806 00:52:54.056958 10445 leader_election.cc:288] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [CANDIDATE]: Term 1 election: Requested vote from peers c03d6a0300ed4445b2217a8dab76b56e (127.15.147.130:33897), ca1a58e190114c7198a21d6f2a1b9354 (127.15.147.131:38591)
I20260806 00:52:54.057401 10150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af973c8fe86545b1988575448a8eb71e" candidate_uuid: "809c1352b75346bdac21294705ea8c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c03d6a0300ed4445b2217a8dab76b56e"
I20260806 00:52:54.057595 10150 raft_consensus.cc:3037] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:54.058234 10341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af973c8fe86545b1988575448a8eb71e" candidate_uuid: "809c1352b75346bdac21294705ea8c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1a58e190114c7198a21d6f2a1b9354"
I20260806 00:52:54.058476 10341 raft_consensus.cc:3037] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:54.059092 10150 raft_consensus.cc:2454] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 809c1352b75346bdac21294705ea8c3a in term 1.
I20260806 00:52:54.059937 9861 leader_election.cc:302] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 809c1352b75346bdac21294705ea8c3a, c03d6a0300ed4445b2217a8dab76b56e; no voters:
I20260806 00:52:54.060288 10341 raft_consensus.cc:2454] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 809c1352b75346bdac21294705ea8c3a in term 1.
I20260806 00:52:54.060964 10445 raft_consensus.cc:2781] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:52:54.061254 10445 raft_consensus.cc:686] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [term 1 LEADER]: Becoming Leader. State: Replica: 809c1352b75346bdac21294705ea8c3a, State: Running, Role: LEADER
I20260806 00:52:54.061631 10445 consensus_queue.cc:237] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c03d6a0300ed4445b2217a8dab76b56e" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33897 } } peers { permanent_uuid: "ca1a58e190114c7198a21d6f2a1b9354" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38591 } } peers { permanent_uuid: "809c1352b75346bdac21294705ea8c3a" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 34079 } }
I20260806 00:52:54.065412 9755 catalog_manager.cc:5495] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 809c1352b75346bdac21294705ea8c3a (127.15.147.129). New cstate: current_term: 1 leader_uuid: "809c1352b75346bdac21294705ea8c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c03d6a0300ed4445b2217a8dab76b56e" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca1a58e190114c7198a21d6f2a1b9354" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "809c1352b75346bdac21294705ea8c3a" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 34079 } health_report { overall_health: HEALTHY } } }
I20260806 00:52:54.102445 10436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:54.099014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42640 (local address 127.15.147.129:34079)
0806 00:52:54.099165 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:54.099169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:54.099184 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:52:54.099273 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:54.099276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:54.099347 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:54.099431 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:52:54.099666 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.100173 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.101003 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.101203 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.101278 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:54.101537 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:52:54.101941 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:52:54.102048 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:54.102188 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 00:52:54.102254 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 00:52:54.108739 10150 raft_consensus.cc:1261] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e [term 1 FOLLOWER]: Refusing update from remote peer 809c1352b75346bdac21294705ea8c3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:52:54.108753 10341 raft_consensus.cc:1261] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354 [term 1 FOLLOWER]: Refusing update from remote peer 809c1352b75346bdac21294705ea8c3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260806 00:52:54.109315 10415 tablet.cc:2369] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:52:54.109512 10452 consensus_queue.cc:1035] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c03d6a0300ed4445b2217a8dab76b56e" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 33897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:52:54.109678 10445 consensus_queue.cc:1035] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca1a58e190114c7198a21d6f2a1b9354" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 38591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:52:54.127235 9755 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:52:54.127444 9755 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:52:54.129880 9755 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f8f73ef50f945c0b5164d97dad3e584: Sending DeleteTablet for 3 replicas of tablet af973c8fe86545b1988575448a8eb71e
I20260806 00:52:54.130710 10117 tablet_service.cc:1514] Processing DeleteTablet for tablet af973c8fe86545b1988575448a8eb71e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:54 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:36580
I20260806 00:52:54.130786 9926 tablet_service.cc:1514] Processing DeleteTablet for tablet af973c8fe86545b1988575448a8eb71e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:54 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:42638
I20260806 00:52:54.131289 10467 tablet_replica.cc:330] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e: stopping tablet replica
I20260806 00:52:54.131378 10315 tablet_service.cc:1514] Processing DeleteTablet for tablet af973c8fe86545b1988575448a8eb71e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:54 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:48992
I20260806 00:52:54.131599 10468 tablet_replica.cc:330] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a: stopping tablet replica
I20260806 00:52:54.131842 10467 raft_consensus.cc:2229] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:52:54.131894 10468 raft_consensus.cc:2229] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:52:54.132273 10467 raft_consensus.cc:2258] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:54.132324 10468 raft_consensus.cc:2258] T af973c8fe86545b1988575448a8eb71e P 809c1352b75346bdac21294705ea8c3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:54.132506 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 9838
I20260806 00:52:54.137972 10469 tablet_replica.cc:330] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354: stopping tablet replica
I20260806 00:52:54.138324 10469 raft_consensus.cc:2229] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:52:54.138718 10469 raft_consensus.cc:2258] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:54.139672 10467 ts_tablet_manager.cc:1905] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:52:54.140527 10469 ts_tablet_manager.cc:1905] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:52:54.142230 10467 ts_tablet_manager.cc:1918] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:52:54.142362 10467 log.cc:1196] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/af973c8fe86545b1988575448a8eb71e
I20260806 00:52:54.142678 10467 ts_tablet_manager.cc:1939] T af973c8fe86545b1988575448a8eb71e P c03d6a0300ed4445b2217a8dab76b56e: Deleting consensus metadata
I20260806 00:52:54.143966 9738 catalog_manager.cc:4841] TS c03d6a0300ed4445b2217a8dab76b56e (127.15.147.130:33897): tablet af973c8fe86545b1988575448a8eb71e (table test-table [id=da329a5f480b49a199466b32a519bc04]) successfully deleted
I20260806 00:52:54.144182 10469 ts_tablet_manager.cc:1918] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:52:54.144337 10469 log.cc:1196] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/af973c8fe86545b1988575448a8eb71e
I20260806 00:52:54.144748 10469 ts_tablet_manager.cc:1939] T af973c8fe86545b1988575448a8eb71e P ca1a58e190114c7198a21d6f2a1b9354: Deleting consensus metadata
I20260806 00:52:54.146131 9737 catalog_manager.cc:4841] TS ca1a58e190114c7198a21d6f2a1b9354 (127.15.147.131:38591): tablet af973c8fe86545b1988575448a8eb71e (table test-table [id=da329a5f480b49a199466b32a519bc04]) successfully deleted
W20260806 00:52:54.147189 9737 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.147.129:34079 (error 108)
I20260806 00:52:54.147471 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 10022
I20260806 00:52:54.148731 10431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:54.148247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:34079 (local address 127.0.0.1:42654)
0806 00:52:54.148519 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:54.148614 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:34079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
W20260806 00:52:54.148940 9737 catalog_manager.cc:4584] TS 809c1352b75346bdac21294705ea8c3a (127.15.147.129:34079): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet af973c8fe86545b1988575448a8eb71e: Network error: Client connection negotiation failed: client connection to 127.15.147.129:34079: connect: Connection refused (error 111)
I20260806 00:52:54.158222 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 10226
I20260806 00:52:54.177049 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 9712
2026-08-06T00:52:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2637 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T00:52:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:52:54Z Disabled control of system clock
I20260806 00:52:54.217233 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:34905
--webserver_interface=127.15.147.190
--webserver_port=0
--builtin_ntp_servers=127.15.147.148:37303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:34905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 00:52:54.340507 10474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:54.340795 10474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:54.340832 10474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:54.344945 10474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:52:54.345007 10474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:54.345022 10474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:54.345039 10474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:52:54.345055 10474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:52:54.349797 10474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:37303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:34905
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:34905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:54.350858 10474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:54.352216 10474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:54.361043 10481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:54.361083 10480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:54.361066 10474 server_base.cc:1034] running on GCE node
W20260806 00:52:54.361256 10483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:54.361903 10474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:54.363065 10474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:54.364269 10474 hybrid_clock.cc:648] HybridClock initialized: now 1785977574364258 us; error 45 us; skew 500 ppm
I20260806 00:52:54.366281 10474 webserver.cc:460] Webserver started at http://127.15.147.190:37693/ using document root <none> and password file <none>
I20260806 00:52:54.366895 10474 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:54.366999 10474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:54.367245 10474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:54.369652 10474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "44e70d845184476092ffd174d4c69ba7"
format_stamp: "Formatted at 2026-08-06 00:52:54 on dist-test-slave-45dx"
server_key: "434a830a326e956e6a36177b8d9871a9"
server_key_iv: "4dc0c6787923ec4e0df392f61d4a8061"
server_key_version: "encryptionkey@0"
I20260806 00:52:54.370265 10474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "44e70d845184476092ffd174d4c69ba7"
format_stamp: "Formatted at 2026-08-06 00:52:54 on dist-test-slave-45dx"
server_key: "434a830a326e956e6a36177b8d9871a9"
server_key_iv: "4dc0c6787923ec4e0df392f61d4a8061"
server_key_version: "encryptionkey@0"
I20260806 00:52:54.374135 10474 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:52:54.376662 10493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:54.377970 10474 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:52:54.378137 10474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "44e70d845184476092ffd174d4c69ba7"
format_stamp: "Formatted at 2026-08-06 00:52:54 on dist-test-slave-45dx"
server_key: "434a830a326e956e6a36177b8d9871a9"
server_key_iv: "4dc0c6787923ec4e0df392f61d4a8061"
server_key_version: "encryptionkey@0"
I20260806 00:52:54.378263 10474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:54.404453 10474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:54.405278 10474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:54.405511 10474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:54.413283 10474 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:34905
I20260806 00:52:54.413297 10565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:34905 every 8 connection(s)
I20260806 00:52:54.414310 10474 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:52:54.417688 10566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:54.423524 10566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7: Bootstrap starting.
I20260806 00:52:54.424544 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 10474
I20260806 00:52:54.424676 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:52:54.425037 15950 external_mini_cluster.cc:1428] Setting key 6960a9201844bf44401c3d51a7b25b83
I20260806 00:52:54.426658 10566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:54.427459 10566 log.cc:824] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:54.429540 10566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7: No bootstrap required, opened a new log
I20260806 00:52:54.433053 10566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44e70d845184476092ffd174d4c69ba7" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34905 } }
I20260806 00:52:54.433301 10566 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:54.433357 10566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44e70d845184476092ffd174d4c69ba7, State: Initialized, Role: FOLLOWER
I20260806 00:52:54.433848 10566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44e70d845184476092ffd174d4c69ba7" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34905 } }
I20260806 00:52:54.434015 10566 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:52:54.434122 10566 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:52:54.434252 10566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:54.434679 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:54.426275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57138 (local address 127.15.147.190:34905)
0806 00:52:54.427032 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:54.427048 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:54.427085 (+ 37us) server_negotiation.cc:407] Connection header received
0806 00:52:54.427824 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:54.427852 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:54.428292 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:54.428693 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:54.429128 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.430144 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.431078 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.431571 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.431751 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:54.432700 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:54.432705 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:54.432718 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:52:54.433106 (+ 388us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:54.433163 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:52:54.433182 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:54.433295 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:54.433412 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 00:52:54.433724 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":141,"threads_started":1}
I20260806 00:52:54.435240 10566 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44e70d845184476092ffd174d4c69ba7" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34905 } }
I20260806 00:52:54.435637 10566 leader_election.cc:302] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44e70d845184476092ffd174d4c69ba7; no voters:
I20260806 00:52:54.436007 10566 leader_election.cc:288] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:52:54.436127 10573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:52:54.436409 10573 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7 [term 1 LEADER]: Becoming Leader. State: Replica: 44e70d845184476092ffd174d4c69ba7, State: Running, Role: LEADER
I20260806 00:52:54.436740 10573 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44e70d845184476092ffd174d4c69ba7" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34905 } }
I20260806 00:52:54.437299 10566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:52:54.437939 10573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44e70d845184476092ffd174d4c69ba7. Latest consensus state: current_term: 1 leader_uuid: "44e70d845184476092ffd174d4c69ba7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44e70d845184476092ffd174d4c69ba7" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34905 } } }
I20260806 00:52:54.438066 10573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7 [sys.catalog]: This master's current role is: LEADER
I20260806 00:52:54.439752 10575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44e70d845184476092ffd174d4c69ba7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44e70d845184476092ffd174d4c69ba7" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34905 } } }
I20260806 00:52:54.439879 10575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7 [sys.catalog]: This master's current role is: LEADER
I20260806 00:52:54.440492 10581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:52:54.443388 10581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:52:54.449258 10581 catalog_manager.cc:1347] Generated new cluster ID: b495b10476184ba999b4be0f7615baa3
I20260806 00:52:54.449333 10581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:52:54.463697 10581 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:52:54.464749 10581 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:52:54.475572 10581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7: Generated new TSK 0
I20260806 00:52:54.476539 10581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:52:54.482805 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:34905
--builtin_ntp_servers=127.15.147.148:37303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 00:52:54.598654 10600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:54.598932 10600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:54.598990 10600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:54.603076 10600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:54.603138 10600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:54.603192 10600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:52:54.607921 10600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:37303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:34905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:54.609089 10600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:54.610543 10600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:54.620390 10606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:54.620445 10607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:54.620456 10600 server_base.cc:1034] running on GCE node
W20260806 00:52:54.620398 10610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:54.621089 10600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:54.621733 10600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:54.622941 10600 hybrid_clock.cc:648] HybridClock initialized: now 1785977574622930 us; error 29 us; skew 500 ppm
I20260806 00:52:54.624806 10600 webserver.cc:460] Webserver started at http://127.15.147.129:34193/ using document root <none> and password file <none>
I20260806 00:52:54.625380 10600 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:54.625485 10600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:54.625723 10600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:54.627748 10600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "c6ca875fe5534220b77eedb4cfe02865"
format_stamp: "Formatted at 2026-08-06 00:52:54 on dist-test-slave-45dx"
server_key: "070bd36668286b0aea58079e8f816b0e"
server_key_iv: "a8e65c5ad1663bc759469e435c1e7436"
server_key_version: "encryptionkey@0"
I20260806 00:52:54.628368 10600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "c6ca875fe5534220b77eedb4cfe02865"
format_stamp: "Formatted at 2026-08-06 00:52:54 on dist-test-slave-45dx"
server_key: "070bd36668286b0aea58079e8f816b0e"
server_key_iv: "a8e65c5ad1663bc759469e435c1e7436"
server_key_version: "encryptionkey@0"
I20260806 00:52:54.631870 10600 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:52:54.634697 10618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:54.635836 10600 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:52:54.635967 10600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "c6ca875fe5534220b77eedb4cfe02865"
format_stamp: "Formatted at 2026-08-06 00:52:54 on dist-test-slave-45dx"
server_key: "070bd36668286b0aea58079e8f816b0e"
server_key_iv: "a8e65c5ad1663bc759469e435c1e7436"
server_key_version: "encryptionkey@0"
I20260806 00:52:54.636087 10600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:54.656651 10600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:54.657366 10600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:54.657599 10600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:54.658217 10600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:54.659229 10600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:54.659307 10600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:54.659384 10600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:54.659426 10600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:54.669035 10600 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:40903
I20260806 00:52:54.669064 10779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:40903 every 8 connection(s)
I20260806 00:52:54.670020 10600 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:52:54.670639 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 10600
I20260806 00:52:54.670776 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:52:54.671103 15950 external_mini_cluster.cc:1428] Setting key 2d21f94c42024120c0722db4a5ab4124
I20260806 00:52:54.673408 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:34905
--builtin_ntp_servers=127.15.147.148:37303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 00:52:54.679723 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:54.672011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:50905 (local address 127.15.147.190:34905)
0806 00:52:54.672153 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:54.672158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:54.673407 (+ 1249us) server_negotiation.cc:407] Connection header received
0806 00:52:54.674934 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:54.674937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:54.674996 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:54.675075 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:54.675969 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.676539 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.677677 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.677928 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.678044 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:54.679226 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:54.679229 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:54.679232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:52:54.679249 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:54.679268 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:52:54.679275 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:54.679350 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:54.679520 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 00:52:54.679583 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:52:54.680639 10782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:54.672543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:34905 (local address 127.15.147.129:50905)
0806 00:52:54.673257 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:54.673298 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:54.674700 (+ 1402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:54.675246 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:54.675253 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:54.675334 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:54.675818 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:54.675830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.676671 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.676675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:54.677526 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:54.677536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.678090 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.678094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:54.678267 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:54.678957 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:52:54.678959 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:54.678998 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:52:54.679073 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:54.679353 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:54.679377 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:54.679389 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 00:52:54.679517 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 00:52:54.679787 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":168,"threads_started":1}
I20260806 00:52:54.681948 10780 heartbeater.cc:347] Connected to a master server at 127.15.147.190:34905
I20260806 00:52:54.682206 10780 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:54.682778 10780 heartbeater.cc:511] Master 127.15.147.190:34905 requested a full tablet report, sending...
I20260806 00:52:54.684371 10515 ts_manager.cc:194] Registered new tserver with Master: c6ca875fe5534220b77eedb4cfe02865 (127.15.147.129:40903)
I20260806 00:52:54.685561 10515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.147.129:50905
W20260806 00:52:54.788712 10783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:54.789003 10783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:54.789100 10783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:54.793130 10783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:54.793200 10783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:54.793286 10783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:52:54.797869 10783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:37303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:34905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:54.798928 10783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:54.800460 10783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:54.809976 10791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:54.810051 10783 server_base.cc:1034] running on GCE node
W20260806 00:52:54.809965 10790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:54.809952 10794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:54.810714 10783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:54.811365 10783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:54.812553 10783 hybrid_clock.cc:648] HybridClock initialized: now 1785977574812541 us; error 24 us; skew 500 ppm
I20260806 00:52:54.814342 10783 webserver.cc:460] Webserver started at http://127.15.147.130:45387/ using document root <none> and password file <none>
I20260806 00:52:54.814911 10783 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:54.815008 10783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:54.815219 10783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:54.817312 10783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "45563b8b155d4e80b4629111c1c138b5"
format_stamp: "Formatted at 2026-08-06 00:52:54 on dist-test-slave-45dx"
server_key: "0e2c28e13d4dface43eee99743d20d7f"
server_key_iv: "352573496b9ad695e5f73e1aff5174b3"
server_key_version: "encryptionkey@0"
I20260806 00:52:54.817889 10783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "45563b8b155d4e80b4629111c1c138b5"
format_stamp: "Formatted at 2026-08-06 00:52:54 on dist-test-slave-45dx"
server_key: "0e2c28e13d4dface43eee99743d20d7f"
server_key_iv: "352573496b9ad695e5f73e1aff5174b3"
server_key_version: "encryptionkey@0"
I20260806 00:52:54.821386 10783 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:52:54.823726 10803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:54.824928 10783 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:52:54.825064 10783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "45563b8b155d4e80b4629111c1c138b5"
format_stamp: "Formatted at 2026-08-06 00:52:54 on dist-test-slave-45dx"
server_key: "0e2c28e13d4dface43eee99743d20d7f"
server_key_iv: "352573496b9ad695e5f73e1aff5174b3"
server_key_version: "encryptionkey@0"
I20260806 00:52:54.825172 10783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:54.846115 10783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:54.846879 10783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:54.847106 10783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:54.847705 10783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:54.848723 10783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:54.848798 10783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:54.848902 10783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:54.848938 10783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:54.857656 10783 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:44177
I20260806 00:52:54.857688 10967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:44177 every 8 connection(s)
I20260806 00:52:54.858520 10783 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:52:54.860878 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 10783
I20260806 00:52:54.861025 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:52:54.861356 15950 external_mini_cluster.cc:1428] Setting key 240602cb1767d0e469c4c3bd69f82755
I20260806 00:52:54.863636 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:34905
--builtin_ntp_servers=127.15.147.148:37303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 00:52:54.869499 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:54.860265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:44693 (local address 127.15.147.190:34905)
0806 00:52:54.860448 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:54.860451 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:54.862258 (+ 1807us) server_negotiation.cc:407] Connection header received
0806 00:52:54.863941 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:54.863944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:54.863997 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:54.864067 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:54.865249 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.866027 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.867164 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.867569 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.867693 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:54.868889 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:54.868897 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:54.868901 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:52:54.868921 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:54.868948 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:52:54.868962 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:54.869058 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:54.869215 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 00:52:54.869291 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 00:52:54.870182 10972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:54.860753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:34905 (local address 127.15.147.130:44693)
0806 00:52:54.862126 (+ 1373us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:54.862166 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:54.863658 (+ 1492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:54.864239 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:54.864250 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:54.864383 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:54.865080 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:54.865109 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.866163 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.866166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:54.867032 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:54.867041 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:54.867715 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:54.867717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:54.867881 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:54.868581 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:52:54.868582 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:54.868620 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:52:54.868711 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:54.869063 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:54.869070 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:54.869074 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 00:52:54.869161 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 00:52:54.869366 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1009,"thread_start_us":137,"threads_started":1}
I20260806 00:52:54.871351 10970 heartbeater.cc:347] Connected to a master server at 127.15.147.190:34905
I20260806 00:52:54.871569 10970 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:54.872073 10970 heartbeater.cc:511] Master 127.15.147.190:34905 requested a full tablet report, sending...
I20260806 00:52:54.873196 10515 ts_manager.cc:194] Registered new tserver with Master: 45563b8b155d4e80b4629111c1c138b5 (127.15.147.130:44177)
I20260806 00:52:54.873781 10515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.147.130:44693
W20260806 00:52:54.978400 10973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:54.978627 10973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:54.978662 10973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:54.982669 10973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:54.982717 10973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:54.982802 10973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:52:54.987334 10973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:37303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:34905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:54.988288 10973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:54.989629 10973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:52:54.999177 10973 server_base.cc:1034] running on GCE node
W20260806 00:52:54.999163 10978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:54.999152 10982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:54.999197 10979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:54.999874 10973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:55.000656 10973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:55.001871 10973 hybrid_clock.cc:648] HybridClock initialized: now 1785977575001855 us; error 26 us; skew 500 ppm
I20260806 00:52:55.003793 10973 webserver.cc:460] Webserver started at http://127.15.147.131:40275/ using document root <none> and password file <none>
I20260806 00:52:55.004509 10973 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:55.004607 10973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:55.004873 10973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:55.007236 10973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "4745069dae6b46f29b58f79ff380315a"
format_stamp: "Formatted at 2026-08-06 00:52:55 on dist-test-slave-45dx"
server_key: "81fb8d8904f515302d146ed3a8ae672b"
server_key_iv: "111575829004a10577df0f4b64d9882c"
server_key_version: "encryptionkey@0"
I20260806 00:52:55.007843 10973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "4745069dae6b46f29b58f79ff380315a"
format_stamp: "Formatted at 2026-08-06 00:52:55 on dist-test-slave-45dx"
server_key: "81fb8d8904f515302d146ed3a8ae672b"
server_key_iv: "111575829004a10577df0f4b64d9882c"
server_key_version: "encryptionkey@0"
I20260806 00:52:55.011608 10973 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260806 00:52:55.014107 10989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:55.015321 10973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:52:55.015483 10973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "4745069dae6b46f29b58f79ff380315a"
format_stamp: "Formatted at 2026-08-06 00:52:55 on dist-test-slave-45dx"
server_key: "81fb8d8904f515302d146ed3a8ae672b"
server_key_iv: "111575829004a10577df0f4b64d9882c"
server_key_version: "encryptionkey@0"
I20260806 00:52:55.015599 10973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:55.030479 10973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:55.031129 10973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:55.031301 10973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:55.031831 10973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:55.032799 10973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:55.032873 10973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:55.032943 10973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:55.032985 10973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:55.042218 10973 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:40999
I20260806 00:52:55.042243 11144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:40999 every 8 connection(s)
I20260806 00:52:55.043071 10973 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:52:55.051397 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 10973
I20260806 00:52:55.051527 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:52:55.051801 15950 external_mini_cluster.cc:1428] Setting key abd1a7a32edf3f1a073e44f982844d01
I20260806 00:52:55.052289 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:55.044927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:32825 (local address 127.15.147.190:34905)
0806 00:52:55.045082 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:55.045085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:55.046088 (+ 1003us) server_negotiation.cc:407] Connection header received
0806 00:52:55.047524 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:55.047527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:55.047586 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:55.047692 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:55.048610 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.049124 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.050271 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.050493 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.050604 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:55.051772 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:55.051776 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:55.051784 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:52:55.051797 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:55.051817 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:52:55.051824 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:55.051901 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:55.052084 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 00:52:55.052147 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 00:52:55.053310 11148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:55.045299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:34905 (local address 127.15.147.131:32825)
0806 00:52:55.045932 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:55.045972 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:55.047303 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:55.047860 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:55.047866 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:55.047945 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:55.048461 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:55.048482 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.049257 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.049260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:55.050140 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:55.050149 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.050648 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.050652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:55.050807 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:55.051446 (+ 639us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:52:55.051448 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:55.051509 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:52:55.051614 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:55.051903 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:55.051915 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:55.051924 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 00:52:55.052078 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 00:52:55.052372 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":128,"threads_started":1}
I20260806 00:52:55.054555 11146 heartbeater.cc:347] Connected to a master server at 127.15.147.190:34905
I20260806 00:52:55.054783 11146 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:55.055296 11146 heartbeater.cc:511] Master 127.15.147.190:34905 requested a full tablet report, sending...
I20260806 00:52:55.056229 10515 ts_manager.cc:194] Registered new tserver with Master: 4745069dae6b46f29b58f79ff380315a (127.15.147.131:40999)
I20260806 00:52:55.056842 10515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.147.131:32825
I20260806 00:52:55.065863 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:52:55.071415 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:55.067849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57154 (local address 127.15.147.190:34905)
0806 00:52:55.067999 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:55.068002 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:55.068190 (+ 188us) server_negotiation.cc:407] Connection header received
0806 00:52:55.068272 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:55.068273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:55.068354 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:55.068425 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:55.068686 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.069184 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.069973 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.070172 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.070245 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:55.071014 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:55.071018 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:55.071020 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:52:55.071031 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:55.071050 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:52:55.071057 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:55.071167 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:55.071277 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 00:52:55.071311 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:52:55.074288 10515 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57154:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:52:55.077289 10515 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:52:55.091655 11161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:55.085466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:40903 (local address 127.0.0.1:50434)
0806 00:52:55.085825 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:55.085878 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:55.086009 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:55.087763 (+ 1754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:55.087766 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:55.087782 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:55.087877 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:55.087883 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.089394 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.089397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:55.090332 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:55.090342 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.091104 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.091108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:55.091208 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:55.091244 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 00:52:55.091368 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 00:52:55.091485 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":55,"thread_start_us":81,"threads_started":1}
I20260806 00:52:55.092046 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:55.085686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:44177 (local address 127.0.0.1:52114)
0806 00:52:55.086567 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:55.086582 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:55.086687 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:55.087515 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:55.087518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:55.087539 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:55.087739 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:55.087745 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.089372 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.089378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:55.090606 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:55.090615 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.091594 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.091597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:55.091690 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:55.091711 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 00:52:55.091777 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 00:52:55.091875 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"mutex_wait_us":122,"thread_start_us":108,"threads_started":1}
I20260806 00:52:55.092077 11166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:55.086140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:40999 (local address 127.0.0.1:56730)
0806 00:52:55.086894 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:55.086908 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:55.087057 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:55.087488 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:55.087494 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:55.087515 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:55.087734 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:55.087744 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.088844 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.088848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:55.090161 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:55.090172 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.091401 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.091405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:55.091490 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:55.091510 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:52:55.091576 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 00:52:55.091627 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":45,"thread_start_us":119,"threads_started":1}
I20260806 00:52:55.092335 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:55.086066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50434 (local address 127.15.147.129:40903)
0806 00:52:55.087323 (+ 1257us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:55.087334 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:55.087356 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:52:55.087433 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:55.087441 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:55.087609 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:55.088040 (+ 431us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:55.088185 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.089281 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.090459 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.090980 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.091117 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:55.091303 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:55.092047 (+ 744us) server_negotiation.cc:299] Negotiation successful
0806 00:52:55.092159 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1150,"thread_start_us":177,"threads_started":1}
I20260806 00:52:55.092464 11169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:55.086394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52114 (local address 127.15.147.130:44177)
0806 00:52:55.086994 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:55.087009 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:55.087036 (+ 27us) server_negotiation.cc:407] Connection header received
0806 00:52:55.087121 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:55.087127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:55.087342 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:55.087711 (+ 369us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:55.087882 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.089223 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.090737 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.091462 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.091595 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:55.091887 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:55.091939 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 00:52:55.092035 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":130,"threads_started":1}
I20260806 00:52:55.092638 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:55.086629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56730 (local address 127.15.147.131:40999)
0806 00:52:55.087001 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:55.087011 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:55.087033 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:52:55.087183 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:55.087187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:55.087329 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:55.087495 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:55.087907 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.088725 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.090300 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.091074 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.091185 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:55.091344 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:55.091733 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 00:52:55.092444 (+ 711us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":148,"threads_started":1}
I20260806 00:52:55.095417 10683 tablet_service.cc:1467] Processing CreateTablet for tablet 8f788a8f96a24360a3f7afc3a0128464 (DEFAULT_TABLE table=test-table [id=839b823e257441a68d31166e0ab77772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:55.095577 11055 tablet_service.cc:1467] Processing CreateTablet for tablet 8f788a8f96a24360a3f7afc3a0128464 (DEFAULT_TABLE table=test-table [id=839b823e257441a68d31166e0ab77772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:55.095417 10865 tablet_service.cc:1467] Processing CreateTablet for tablet 8f788a8f96a24360a3f7afc3a0128464 (DEFAULT_TABLE table=test-table [id=839b823e257441a68d31166e0ab77772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:55.096539 10865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f788a8f96a24360a3f7afc3a0128464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:55.096540 10683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f788a8f96a24360a3f7afc3a0128464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:55.096541 11055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f788a8f96a24360a3f7afc3a0128464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:55.102965 11173 tablet_bootstrap.cc:492] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a: Bootstrap starting.
I20260806 00:52:55.104423 11174 tablet_bootstrap.cc:492] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5: Bootstrap starting.
I20260806 00:52:55.104669 11175 tablet_bootstrap.cc:492] T 8f788a8f96a24360a3f7afc3a0128464 P c6ca875fe5534220b77eedb4cfe02865: Bootstrap starting.
I20260806 00:52:55.105567 11173 tablet_bootstrap.cc:654] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:55.106283 11174 tablet_bootstrap.cc:654] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:55.106513 11173 log.cc:824] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:55.106734 11175 tablet_bootstrap.cc:654] T 8f788a8f96a24360a3f7afc3a0128464 P c6ca875fe5534220b77eedb4cfe02865: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:55.107154 11174 log.cc:824] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:55.107514 11175 log.cc:824] T 8f788a8f96a24360a3f7afc3a0128464 P c6ca875fe5534220b77eedb4cfe02865: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:55.108927 11173 tablet_bootstrap.cc:492] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a: No bootstrap required, opened a new log
I20260806 00:52:55.109150 11173 ts_tablet_manager.cc:1397] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:52:55.109185 11174 tablet_bootstrap.cc:492] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5: No bootstrap required, opened a new log
I20260806 00:52:55.109388 11174 ts_tablet_manager.cc:1397] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:52:55.109781 11175 tablet_bootstrap.cc:492] T 8f788a8f96a24360a3f7afc3a0128464 P c6ca875fe5534220b77eedb4cfe02865: No bootstrap required, opened a new log
I20260806 00:52:55.109989 11175 ts_tablet_manager.cc:1397] T 8f788a8f96a24360a3f7afc3a0128464 P c6ca875fe5534220b77eedb4cfe02865: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 00:52:55.112735 11174 raft_consensus.cc:348] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ca875fe5534220b77eedb4cfe02865" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40903 } } peers { permanent_uuid: "45563b8b155d4e80b4629111c1c138b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44177 } } peers { permanent_uuid: "4745069dae6b46f29b58f79ff380315a" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 40999 } }
I20260806 00:52:55.112972 11174 raft_consensus.cc:374] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:55.113034 11174 raft_consensus.cc:729] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45563b8b155d4e80b4629111c1c138b5, State: Initialized, Role: FOLLOWER
I20260806 00:52:55.113504 11174 consensus_queue.cc:260] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ca875fe5534220b77eedb4cfe02865" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40903 } } peers { permanent_uuid: "45563b8b155d4e80b4629111c1c138b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44177 } } peers { permanent_uuid: "4745069dae6b46f29b58f79ff380315a" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 40999 } }
I20260806 00:52:55.113684 11173 raft_consensus.cc:348] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ca875fe5534220b77eedb4cfe02865" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40903 } } peers { permanent_uuid: "45563b8b155d4e80b4629111c1c138b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44177 } } peers { permanent_uuid: "4745069dae6b46f29b58f79ff380315a" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 40999 } }
I20260806 00:52:55.113925 11173 raft_consensus.cc:374] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:55.113997 11173 raft_consensus.cc:729] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4745069dae6b46f29b58f79ff380315a, State: Initialized, Role: FOLLOWER
I20260806 00:52:55.114171 11175 raft_consensus.cc:348] T 8f788a8f96a24360a3f7afc3a0128464 P c6ca875fe5534220b77eedb4cfe02865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ca875fe5534220b77eedb4cfe02865" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40903 } } peers { permanent_uuid: "45563b8b155d4e80b4629111c1c138b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44177 } } peers { permanent_uuid: "4745069dae6b46f29b58f79ff380315a" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 40999 } }
I20260806 00:52:55.114398 11175 raft_consensus.cc:374] T 8f788a8f96a24360a3f7afc3a0128464 P c6ca875fe5534220b77eedb4cfe02865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:55.114414 10970 heartbeater.cc:503] Master 127.15.147.190:34905 was elected leader, sending a full tablet report...
I20260806 00:52:55.114398 11173 consensus_queue.cc:260] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ca875fe5534220b77eedb4cfe02865" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40903 } } peers { permanent_uuid: "45563b8b155d4e80b4629111c1c138b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44177 } } peers { permanent_uuid: "4745069dae6b46f29b58f79ff380315a" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 40999 } }
I20260806 00:52:55.114964 11175 raft_consensus.cc:729] T 8f788a8f96a24360a3f7afc3a0128464 P c6ca875fe5534220b77eedb4cfe02865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6ca875fe5534220b77eedb4cfe02865, State: Initialized, Role: FOLLOWER
I20260806 00:52:55.115378 11173 ts_tablet_manager.cc:1428] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:52:55.115443 11175 consensus_queue.cc:260] T 8f788a8f96a24360a3f7afc3a0128464 P c6ca875fe5534220b77eedb4cfe02865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ca875fe5534220b77eedb4cfe02865" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40903 } } peers { permanent_uuid: "45563b8b155d4e80b4629111c1c138b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44177 } } peers { permanent_uuid: "4745069dae6b46f29b58f79ff380315a" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 40999 } }
I20260806 00:52:55.115741 11146 heartbeater.cc:503] Master 127.15.147.190:34905 was elected leader, sending a full tablet report...
I20260806 00:52:55.116055 10780 heartbeater.cc:503] Master 127.15.147.190:34905 was elected leader, sending a full tablet report...
I20260806 00:52:55.116410 11175 ts_tablet_manager.cc:1428] T 8f788a8f96a24360a3f7afc3a0128464 P c6ca875fe5534220b77eedb4cfe02865: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 00:52:55.117980 11174 ts_tablet_manager.cc:1428] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260806 00:52:55.149894 11181 raft_consensus.cc:482] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:52:55.150079 11181 raft_consensus.cc:504] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ca875fe5534220b77eedb4cfe02865" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40903 } } peers { permanent_uuid: "45563b8b155d4e80b4629111c1c138b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44177 } } peers { permanent_uuid: "4745069dae6b46f29b58f79ff380315a" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 40999 } }
I20260806 00:52:55.151382 11181 leader_election.cc:288] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6ca875fe5534220b77eedb4cfe02865 (127.15.147.129:40903), 45563b8b155d4e80b4629111c1c138b5 (127.15.147.130:44177)
I20260806 00:52:55.154649 11164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:55.151591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:43537 (local address 127.15.147.129:40903)
0806 00:52:55.151722 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:55.151729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:55.151767 (+ 38us) server_negotiation.cc:407] Connection header received
0806 00:52:55.151826 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:55.151830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:55.151882 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:55.151948 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:55.152248 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.152740 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.153717 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.154114 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.154241 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:55.154290 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:55.154464 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 00:52:55.154529 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:52:55.154649 11148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:55.151473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:40903 (local address 127.15.147.131:43537)
0806 00:52:55.151591 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:55.151608 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:55.151695 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:55.151980 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:55.151983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:55.152000 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:55.152117 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:55.152122 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.152854 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.152857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:55.153598 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:55.153606 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.154236 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.154239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:55.154343 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:55.154362 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 00:52:55.154447 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 00:52:55.154525 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 00:52:55.155226 11169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:55.151766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:58173 (local address 127.15.147.130:44177)
0806 00:52:55.151907 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:55.151911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:55.152126 (+ 215us) server_negotiation.cc:407] Connection header received
0806 00:52:55.152270 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:55.152275 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:55.152378 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:55.152461 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:55.152742 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.153187 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.154140 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.154522 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.154649 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:55.154694 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:55.155052 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 00:52:55.155119 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 00:52:55.155236 11183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:55.151648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:44177 (local address 127.15.147.131:58173)
0806 00:52:55.152025 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:55.152036 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:55.152161 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:55.152475 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:55.152478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:55.152494 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:55.152633 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:55.152638 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.153292 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.153296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:55.154012 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:55.154020 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.154649 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.154652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:55.154904 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:55.154924 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 00:52:55.155020 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 00:52:55.155085 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":139,"threads_started":1}
I20260806 00:52:55.155267 10709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f788a8f96a24360a3f7afc3a0128464" candidate_uuid: "4745069dae6b46f29b58f79ff380315a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6ca875fe5534220b77eedb4cfe02865" is_pre_election: true
I20260806 00:52:55.155582 10709 raft_consensus.cc:2454] T 8f788a8f96a24360a3f7afc3a0128464 P c6ca875fe5534220b77eedb4cfe02865 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4745069dae6b46f29b58f79ff380315a in term 0.
I20260806 00:52:55.155961 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f788a8f96a24360a3f7afc3a0128464" candidate_uuid: "4745069dae6b46f29b58f79ff380315a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45563b8b155d4e80b4629111c1c138b5" is_pre_election: true
I20260806 00:52:55.156066 10993 leader_election.cc:302] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4745069dae6b46f29b58f79ff380315a, c6ca875fe5534220b77eedb4cfe02865; no voters:
I20260806 00:52:55.156265 10898 raft_consensus.cc:2454] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4745069dae6b46f29b58f79ff380315a in term 0.
I20260806 00:52:55.156441 11181 raft_consensus.cc:2781] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:52:55.156533 11181 raft_consensus.cc:482] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:52:55.156564 11181 raft_consensus.cc:3037] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:55.157658 11181 raft_consensus.cc:504] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ca875fe5534220b77eedb4cfe02865" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40903 } } peers { permanent_uuid: "45563b8b155d4e80b4629111c1c138b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44177 } } peers { permanent_uuid: "4745069dae6b46f29b58f79ff380315a" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 40999 } }
I20260806 00:52:55.157994 11181 leader_election.cc:288] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [CANDIDATE]: Term 1 election: Requested vote from peers c6ca875fe5534220b77eedb4cfe02865 (127.15.147.129:40903), 45563b8b155d4e80b4629111c1c138b5 (127.15.147.130:44177)
I20260806 00:52:55.158267 10709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f788a8f96a24360a3f7afc3a0128464" candidate_uuid: "4745069dae6b46f29b58f79ff380315a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6ca875fe5534220b77eedb4cfe02865"
I20260806 00:52:55.158339 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f788a8f96a24360a3f7afc3a0128464" candidate_uuid: "4745069dae6b46f29b58f79ff380315a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45563b8b155d4e80b4629111c1c138b5"
I20260806 00:52:55.158382 10709 raft_consensus.cc:3037] T 8f788a8f96a24360a3f7afc3a0128464 P c6ca875fe5534220b77eedb4cfe02865 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:55.158456 10898 raft_consensus.cc:3037] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:55.159448 10709 raft_consensus.cc:2454] T 8f788a8f96a24360a3f7afc3a0128464 P c6ca875fe5534220b77eedb4cfe02865 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4745069dae6b46f29b58f79ff380315a in term 1.
I20260806 00:52:55.159502 10898 raft_consensus.cc:2454] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4745069dae6b46f29b58f79ff380315a in term 1.
I20260806 00:52:55.159754 10993 leader_election.cc:302] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4745069dae6b46f29b58f79ff380315a, c6ca875fe5534220b77eedb4cfe02865; no voters:
I20260806 00:52:55.159929 11181 raft_consensus.cc:2781] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:52:55.160151 11181 raft_consensus.cc:686] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [term 1 LEADER]: Becoming Leader. State: Replica: 4745069dae6b46f29b58f79ff380315a, State: Running, Role: LEADER
I20260806 00:52:55.160511 11181 consensus_queue.cc:237] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ca875fe5534220b77eedb4cfe02865" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40903 } } peers { permanent_uuid: "45563b8b155d4e80b4629111c1c138b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44177 } } peers { permanent_uuid: "4745069dae6b46f29b58f79ff380315a" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 40999 } }
I20260806 00:52:55.163698 10514 catalog_manager.cc:5495] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4745069dae6b46f29b58f79ff380315a (127.15.147.131). New cstate: current_term: 1 leader_uuid: "4745069dae6b46f29b58f79ff380315a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ca875fe5534220b77eedb4cfe02865" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45563b8b155d4e80b4629111c1c138b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4745069dae6b46f29b58f79ff380315a" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 40999 } health_report { overall_health: HEALTHY } } }
W20260806 00:52:55.171191 10781 tablet.cc:2369] T 8f788a8f96a24360a3f7afc3a0128464 P c6ca875fe5534220b77eedb4cfe02865: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:52:55.189657 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:55.186499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56744 (local address 127.15.147.131:40999)
0806 00:52:55.186619 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:55.186622 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:55.186636 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:52:55.186703 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:55.186706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:55.186757 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:55.186838 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:52:55.187081 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.187560 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.188418 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:55.188622 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:55.188692 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:55.188955 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:52:55.189331 (+ 376us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:52:55.189428 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:55.189510 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 00:52:55.189538 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 00:52:55.195284 10709 raft_consensus.cc:1261] T 8f788a8f96a24360a3f7afc3a0128464 P c6ca875fe5534220b77eedb4cfe02865 [term 1 FOLLOWER]: Refusing update from remote peer 4745069dae6b46f29b58f79ff380315a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:52:55.195350 10898 raft_consensus.cc:1261] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5 [term 1 FOLLOWER]: Refusing update from remote peer 4745069dae6b46f29b58f79ff380315a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:52:55.196046 11181 consensus_queue.cc:1035] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6ca875fe5534220b77eedb4cfe02865" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:52:55.196264 11185 consensus_queue.cc:1035] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [LEADER]: Connected to new peer: Peer: permanent_uuid: "45563b8b155d4e80b4629111c1c138b5" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:52:55.204438 11194 mvcc.cc:205] Tried to move back new op lower bound from 7315364147992944640 to 7315364147860094976. Current Snapshot: MvccSnapshot[applied={T|T < 7315364147992944640}]
I20260806 00:52:55.204622 11192 mvcc.cc:205] Tried to move back new op lower bound from 7315364147992944640 to 7315364147860094976. Current Snapshot: MvccSnapshot[applied={T|T < 7315364147992944640}]
I20260806 00:52:55.212118 10515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:52:55.212249 10515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:52:55.214407 10515 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44e70d845184476092ffd174d4c69ba7: Sending DeleteTablet for 3 replicas of tablet 8f788a8f96a24360a3f7afc3a0128464
I20260806 00:52:55.215078 10683 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f788a8f96a24360a3f7afc3a0128464 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:55 UTC) from {username='slave'} at 127.0.0.1:50434
I20260806 00:52:55.215217 11055 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f788a8f96a24360a3f7afc3a0128464 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:55 UTC) from {username='slave'} at 127.0.0.1:56730
I20260806 00:52:55.215583 11206 tablet_replica.cc:330] T 8f788a8f96a24360a3f7afc3a0128464 P c6ca875fe5534220b77eedb4cfe02865: stopping tablet replica
I20260806 00:52:55.215623 10865 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f788a8f96a24360a3f7afc3a0128464 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:55 UTC) from {username='slave'} at 127.0.0.1:52114
I20260806 00:52:55.215999 11206 raft_consensus.cc:2229] T 8f788a8f96a24360a3f7afc3a0128464 P c6ca875fe5534220b77eedb4cfe02865 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:52:55.216064 11208 tablet_replica.cc:330] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5: stopping tablet replica
I20260806 00:52:55.216287 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 10600
I20260806 00:52:55.216380 11208 raft_consensus.cc:2229] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:52:55.216537 11206 raft_consensus.cc:2258] T 8f788a8f96a24360a3f7afc3a0128464 P c6ca875fe5534220b77eedb4cfe02865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:55.219117 11208 raft_consensus.cc:2258] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:55.221292 11208 ts_tablet_manager.cc:1905] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:52:55.224232 11208 ts_tablet_manager.cc:1918] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:52:55.224367 11208 log.cc:1196] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/8f788a8f96a24360a3f7afc3a0128464
I20260806 00:52:55.224658 11207 tablet_replica.cc:330] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a: stopping tablet replica
I20260806 00:52:55.224707 11208 ts_tablet_manager.cc:1939] T 8f788a8f96a24360a3f7afc3a0128464 P 45563b8b155d4e80b4629111c1c138b5: Deleting consensus metadata
I20260806 00:52:55.225070 11207 raft_consensus.cc:2229] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:52:55.225525 11207 raft_consensus.cc:2258] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:55.225919 10496 catalog_manager.cc:4841] TS 45563b8b155d4e80b4629111c1c138b5 (127.15.147.130:44177): tablet 8f788a8f96a24360a3f7afc3a0128464 (table test-table [id=839b823e257441a68d31166e0ab77772]) successfully deleted
I20260806 00:52:55.227170 11207 ts_tablet_manager.cc:1905] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 00:52:55.229642 10497 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.147.129:40903 (error 108)
I20260806 00:52:55.230406 11207 ts_tablet_manager.cc:1918] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:52:55.230504 11207 log.cc:1196] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/8f788a8f96a24360a3f7afc3a0128464
I20260806 00:52:55.230500 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 10783
I20260806 00:52:55.230654 11166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:55.230376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:40903 (local address 127.0.0.1:50438)
0806 00:52:55.230488 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:55.230564 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:40903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 00:52:55.232560 11207 ts_tablet_manager.cc:1939] T 8f788a8f96a24360a3f7afc3a0128464 P 4745069dae6b46f29b58f79ff380315a: Deleting consensus metadata
W20260806 00:52:55.232558 10497 catalog_manager.cc:4584] TS c6ca875fe5534220b77eedb4cfe02865 (127.15.147.129:40903): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8f788a8f96a24360a3f7afc3a0128464: Network error: Client connection negotiation failed: client connection to 127.15.147.129:40903: connect: Connection refused (error 111)
I20260806 00:52:55.233738 10496 catalog_manager.cc:4841] TS 4745069dae6b46f29b58f79ff380315a (127.15.147.131:40999): tablet 8f788a8f96a24360a3f7afc3a0128464 (table test-table [id=839b823e257441a68d31166e0ab77772]) successfully deleted
I20260806 00:52:55.240901 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 10973
I20260806 00:52:55.253331 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 10474
2026-08-06T00:52:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1073 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:52:55 dist-test-slave-45dx krb5kdc[11212](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:52:55 dist-test-slave-45dx krb5kdc[11212](info): set up 2 sockets
Aug 06 00:52:55 dist-test-slave-45dx krb5kdc[11212](info): commencing operation
krb5kdc: starting...
W20260806 00:52:56.467572 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:52:56 dist-test-slave-45dx krb5kdc[11212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:52:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:52:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:52:56.595314 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:33845
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:45215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:33845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:52:56.713598 11235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:56.713856 11235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:56.713896 11235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:56.718050 11235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:52:56.718132 11235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:56.718153 11235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:56.718192 11235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:52:56.718225 11235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:52:56.723572 11235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:45215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:33845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:33845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:56.724792 11235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:56.725845 11235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:56.734428 11242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:56.734452 11244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:56.734623 11235 server_base.cc:1034] running on GCE node
W20260806 00:52:56.734472 11241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:56.735236 11235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:56.736231 11235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:56.737402 11235 hybrid_clock.cc:648] HybridClock initialized: now 1785977576737373 us; error 42 us; skew 500 ppm
Aug 06 00:52:56 dist-test-slave-45dx krb5kdc[11212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977576, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:56.740437 11235 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:52:56.741282 11235 webserver.cc:460] Webserver started at http://127.15.147.190:36647/ using document root <none> and password file <none>
I20260806 00:52:56.741838 11235 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:56.741900 11235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:56.742065 11235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:56.744091 11235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "d6e75b5c9140472b9fe4fffe4e03f63d"
format_stamp: "Formatted at 2026-08-06 00:52:56 on dist-test-slave-45dx"
server_key: "31d247d193a90be609804876e0e270a5"
server_key_iv: "e95c8c513cdb1e16c05d61bb617a23fa"
server_key_version: "encryptionkey@0"
I20260806 00:52:56.744721 11235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "d6e75b5c9140472b9fe4fffe4e03f63d"
format_stamp: "Formatted at 2026-08-06 00:52:56 on dist-test-slave-45dx"
server_key: "31d247d193a90be609804876e0e270a5"
server_key_iv: "e95c8c513cdb1e16c05d61bb617a23fa"
server_key_version: "encryptionkey@0"
I20260806 00:52:56.748579 11235 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 00:52:56.750953 11254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:56.752166 11235 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:52:56.752362 11235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "d6e75b5c9140472b9fe4fffe4e03f63d"
format_stamp: "Formatted at 2026-08-06 00:52:56 on dist-test-slave-45dx"
server_key: "31d247d193a90be609804876e0e270a5"
server_key_iv: "e95c8c513cdb1e16c05d61bb617a23fa"
server_key_version: "encryptionkey@0"
I20260806 00:52:56.752471 11235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:56.770923 11235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:56.774631 11235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:56.774868 11235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:56.782399 11235 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:33845
I20260806 00:52:56.782418 11329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:33845 every 8 connection(s)
I20260806 00:52:56.783354 11235 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:52:56.786306 11330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:56.791842 11330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d: Bootstrap starting.
I20260806 00:52:56.792738 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 11235
I20260806 00:52:56.792874 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:52:56.793227 15950 external_mini_cluster.cc:1428] Setting key 1bf86dfbb98321cc23aa625ccac85a8f
I20260806 00:52:56.794972 11330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:56.795735 11330 log.cc:824] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:56.797736 11330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d: No bootstrap required, opened a new log
I20260806 00:52:56.801339 11330 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e75b5c9140472b9fe4fffe4e03f63d" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33845 } }
I20260806 00:52:56.801553 11330 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:56.801589 11330 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6e75b5c9140472b9fe4fffe4e03f63d, State: Initialized, Role: FOLLOWER
I20260806 00:52:56.802076 11330 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e75b5c9140472b9fe4fffe4e03f63d" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33845 } }
Aug 06 00:52:56 dist-test-slave-45dx krb5kdc[11212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:52:56.802217 11330 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:52:56.802321 11330 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:52:56.802441 11330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:56.803763 11330 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e75b5c9140472b9fe4fffe4e03f63d" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33845 } }
I20260806 00:52:56.804158 11330 leader_election.cc:302] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6e75b5c9140472b9fe4fffe4e03f63d; no voters:
I20260806 00:52:56.804566 11330 leader_election.cc:288] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:52:56.804643 11337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:52:56.804859 11337 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d [term 1 LEADER]: Becoming Leader. State: Replica: d6e75b5c9140472b9fe4fffe4e03f63d, State: Running, Role: LEADER
I20260806 00:52:56.805161 11337 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e75b5c9140472b9fe4fffe4e03f63d" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33845 } }
I20260806 00:52:56.805919 11330 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:52:56.806360 11337 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6e75b5c9140472b9fe4fffe4e03f63d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e75b5c9140472b9fe4fffe4e03f63d" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33845 } } }
I20260806 00:52:56.806489 11337 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d [sys.catalog]: This master's current role is: LEADER
I20260806 00:52:56.806895 11339 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6e75b5c9140472b9fe4fffe4e03f63d. Latest consensus state: current_term: 1 leader_uuid: "d6e75b5c9140472b9fe4fffe4e03f63d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6e75b5c9140472b9fe4fffe4e03f63d" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33845 } } }
I20260806 00:52:56.807008 11339 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d [sys.catalog]: This master's current role is: LEADER
I20260806 00:52:56.807312 11348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:52:56.809088 11334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:56.794447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59680 (local address 127.15.147.190:33845)
0806 00:52:56.794908 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:56.794923 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:56.794961 (+ 38us) server_negotiation.cc:407] Connection header received
0806 00:52:56.795678 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:56.795695 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:56.796091 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:56.796482 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:56.797352 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:56.798361 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:56.799265 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:56.799746 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:56.799919 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:56.802879 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:56.802885 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:56.802902 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:56.802936 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:56.805164 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:56.805696 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:56.805702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:56.805708 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:56.805840 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:56.806185 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:56.806189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:56.806190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:56.806948 (+ 758us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:56.807151 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:56.807662 (+ 511us) server_negotiation.cc:299] Negotiation successful
0806 00:52:56.808022 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":122,"threads_started":1}
I20260806 00:52:56.810750 11348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:52:56.816813 11348 catalog_manager.cc:1347] Generated new cluster ID: d6a466ea0cce4cc9b067749e99cc3b6b
I20260806 00:52:56.816891 11348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:52:56.837001 11348 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:52:56.838017 11348 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:52:56.842190 11348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d: Generated new TSK 0
I20260806 00:52:56.842883 11348 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.147.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
Entry for principal kudu/127.15.147.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab.
I20260806 00:52:56.885290 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:33845
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:45215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:52:57.025596 11372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:57.025885 11372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:57.025957 11372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:57.030145 11372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:57.030234 11372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:57.030328 11372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:52:57.035612 11372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:45215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.129.keytab
--principal=kudu/127.15.147.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:33845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:57.036664 11372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:57.037674 11372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:57.047493 11383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:57.047497 11380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:57.047493 11381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:57.048017 11372 server_base.cc:1034] running on GCE node
I20260806 00:52:57.048473 11372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:57.049098 11372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:57.050292 11372 hybrid_clock.cc:648] HybridClock initialized: now 1785977577050242 us; error 50 us; skew 500 ppm
Aug 06 00:52:57 dist-test-slave-45dx krb5kdc[11212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977577, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:57.053205 11372 init.cc:377] Logged in from keytab as kudu/127.15.147.129@KRBTEST.COM (short username kudu)
I20260806 00:52:57.054113 11372 webserver.cc:460] Webserver started at http://127.15.147.129:43469/ using document root <none> and password file <none>
I20260806 00:52:57.054733 11372 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:57.054872 11372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:57.055258 11372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:57.057547 11372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "60fb4fb164ca4889921e8014f7ab9fbc"
format_stamp: "Formatted at 2026-08-06 00:52:57 on dist-test-slave-45dx"
server_key: "4dd7a7ade4cb1cce10f9dac3d583305d"
server_key_iv: "3fc3534900b6fb2064cc17478a25623d"
server_key_version: "encryptionkey@0"
I20260806 00:52:57.058120 11372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "60fb4fb164ca4889921e8014f7ab9fbc"
format_stamp: "Formatted at 2026-08-06 00:52:57 on dist-test-slave-45dx"
server_key: "4dd7a7ade4cb1cce10f9dac3d583305d"
server_key_iv: "3fc3534900b6fb2064cc17478a25623d"
server_key_version: "encryptionkey@0"
I20260806 00:52:57.061843 11372 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:52:57.064256 11392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:57.065495 11372 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:52:57.065636 11372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "60fb4fb164ca4889921e8014f7ab9fbc"
format_stamp: "Formatted at 2026-08-06 00:52:57 on dist-test-slave-45dx"
server_key: "4dd7a7ade4cb1cce10f9dac3d583305d"
server_key_iv: "3fc3534900b6fb2064cc17478a25623d"
server_key_version: "encryptionkey@0"
I20260806 00:52:57.065750 11372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:57.084671 11372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:57.088016 11372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:57.088202 11372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:57.088852 11372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:57.089710 11372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:57.089772 11372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:57.089811 11372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:57.089854 11372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:57.098919 11372 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:38805
I20260806 00:52:57.099020 11554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:38805 every 8 connection(s)
I20260806 00:52:57.099896 11372 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:52:57.103979 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 11372
I20260806 00:52:57.104116 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:52:57.104478 15950 external_mini_cluster.cc:1428] Setting key 67fd8d87cee136e43ad3f0e9ffa91a77
Aug 06 00:52:57 dist-test-slave-45dx krb5kdc[11212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977577, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.129@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.147.130@KRBTEST.COM; defaulting to no policy
I20260806 00:52:57.115444 11334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.101890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:47089 (local address 127.15.147.190:33845)
0806 00:52:57.102030 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:57.102034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:57.102870 (+ 836us) server_negotiation.cc:407] Connection header received
0806 00:52:57.104186 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:57.104190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:57.104246 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:57.104442 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:57.105799 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.106399 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.108358 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.108758 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.108885 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:57.111616 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:57.111625 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:57.111627 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:57.111650 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:57.113312 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:57.113851 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:57.113854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:57.113856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:57.113940 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:57.114317 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:57.114320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:57.114322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:57.114508 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:57.114613 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:57.115123 (+ 510us) server_negotiation.cc:299] Negotiation successful
0806 00:52:57.115264 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 00:52:57.116258 11560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.102279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:33845 (local address 127.15.147.129:47089)
0806 00:52:57.102709 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:57.102752 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:57.103963 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:57.104625 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:57.104632 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:57.105033 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:57.105636 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:57.105652 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.106645 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.106652 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:57.108168 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:57.108185 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.108923 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.108927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:57.109078 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:57.109721 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:52:57.109750 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:57.111451 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:57.113440 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:57.113448 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:57.113462 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:57.113734 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:57.114044 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:57.114047 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:57.114049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:57.114195 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:57.114629 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:57.114637 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:57.114888 (+ 251us) client_negotiation.cc:769] Sending connection context
0806 00:52:57.115079 (+ 191us) client_negotiation.cc:240] Negotiation successful
0806 00:52:57.115421 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":104,"threads_started":1}
I20260806 00:52:57.117663 11555 heartbeater.cc:347] Connected to a master server at 127.15.147.190:33845
I20260806 00:52:57.117954 11555 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.130@KRBTEST.COM" created.
I20260806 00:52:57.118551 11555 heartbeater.cc:511] Master 127.15.147.190:33845 requested a full tablet report, sending...
I20260806 00:52:57.120205 11279 ts_manager.cc:194] Registered new tserver with Master: 60fb4fb164ca4889921e8014f7ab9fbc (127.15.147.129:38805)
I20260806 00:52:57.121390 11279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.129@KRBTEST.COM'} at 127.15.147.129:47089
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
Entry for principal kudu/127.15.147.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab.
I20260806 00:52:57.134927 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:33845
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:45215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:52:57.248579 11563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:57.248832 11563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:57.248875 11563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:57.252961 11563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:57.253013 11563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:57.253068 11563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:52:57.258299 11563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:45215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.130.keytab
--principal=kudu/127.15.147.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:33845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:57.259228 11563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:57.260152 11563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:52:57.269704 11563 server_base.cc:1034] running on GCE node
W20260806 00:52:57.269662 11572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:57.269626 11569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:57.269666 11568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:57.270328 11563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:57.270877 11563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:57.272042 11563 hybrid_clock.cc:648] HybridClock initialized: now 1785977577272009 us; error 45 us; skew 500 ppm
Aug 06 00:52:57 dist-test-slave-45dx krb5kdc[11212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977577, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:57.274645 11563 init.cc:377] Logged in from keytab as kudu/127.15.147.130@KRBTEST.COM (short username kudu)
I20260806 00:52:57.275462 11563 webserver.cc:460] Webserver started at http://127.15.147.130:40411/ using document root <none> and password file <none>
I20260806 00:52:57.275970 11563 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:57.276028 11563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:57.276187 11563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:57.278210 11563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "7de4a17d91f747a184f14341f4539f2c"
format_stamp: "Formatted at 2026-08-06 00:52:57 on dist-test-slave-45dx"
server_key: "ebcacdf973ebc7147a2ba5326bfa262f"
server_key_iv: "d8c4c4ad645b3da416e9c1bbd1d4e200"
server_key_version: "encryptionkey@0"
I20260806 00:52:57.278702 11563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "7de4a17d91f747a184f14341f4539f2c"
format_stamp: "Formatted at 2026-08-06 00:52:57 on dist-test-slave-45dx"
server_key: "ebcacdf973ebc7147a2ba5326bfa262f"
server_key_iv: "d8c4c4ad645b3da416e9c1bbd1d4e200"
server_key_version: "encryptionkey@0"
I20260806 00:52:57.282573 11563 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 00:52:57.284919 11579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:57.286135 11563 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:52:57.286259 11563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "7de4a17d91f747a184f14341f4539f2c"
format_stamp: "Formatted at 2026-08-06 00:52:57 on dist-test-slave-45dx"
server_key: "ebcacdf973ebc7147a2ba5326bfa262f"
server_key_iv: "d8c4c4ad645b3da416e9c1bbd1d4e200"
server_key_version: "encryptionkey@0"
I20260806 00:52:57.286365 11563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:57.302536 11563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:57.305420 11563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:57.305619 11563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:57.306314 11563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:57.307350 11563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:57.307404 11563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:57.307471 11563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:57.307507 11563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:57.316486 11563 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:44285
I20260806 00:52:57.316509 11746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:44285 every 8 connection(s)
I20260806 00:52:57.317392 11563 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:52:57.322377 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 11563
I20260806 00:52:57.322508 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:52:57.322799 15950 external_mini_cluster.cc:1428] Setting key c1e0e7d359c1ed3e50018f1841d00c05
Aug 06 00:52:57 dist-test-slave-45dx krb5kdc[11212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977577, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.130@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:52:57.332001 11334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.319080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:59807 (local address 127.15.147.190:33845)
0806 00:52:57.319252 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:57.319257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:57.320218 (+ 961us) server_negotiation.cc:407] Connection header received
0806 00:52:57.321574 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:57.321577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:57.321622 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:57.321696 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:57.322918 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.323574 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.324917 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.325130 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.325241 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:57.328107 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:57.328113 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:57.328119 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:57.328154 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:57.329824 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:57.330421 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:57.330427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:57.330430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:57.330530 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:57.330862 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:57.330866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:57.330867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:57.331078 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:57.331188 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:57.331692 (+ 504us) server_negotiation.cc:299] Negotiation successful
0806 00:52:57.331815 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:52:57.332824 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.319519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:33845 (local address 127.15.147.130:59807)
0806 00:52:57.320059 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:57.320101 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:57.321376 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:57.321867 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:57.321873 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:57.322201 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:57.322763 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:57.322778 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.323743 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.323746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:57.324787 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:57.324798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.325287 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.325291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:57.325490 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:57.326104 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:52:57.326120 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:57.327904 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:57.329957 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:57.329961 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:57.329971 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:57.330310 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:57.330629 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:57.330631 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:57.330633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:57.330763 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:57.331189 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:57.331197 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:57.331439 (+ 242us) client_negotiation.cc:769] Sending connection context
0806 00:52:57.331649 (+ 210us) client_negotiation.cc:240] Negotiation successful
0806 00:52:57.331964 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":109,"threads_started":1}
I20260806 00:52:57.333972 11747 heartbeater.cc:347] Connected to a master server at 127.15.147.190:33845
I20260806 00:52:57.334246 11747 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:57.334837 11747 heartbeater.cc:511] Master 127.15.147.190:33845 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.147.131@KRBTEST.COM; defaulting to no policy
I20260806 00:52:57.336138 11279 ts_manager.cc:194] Registered new tserver with Master: 7de4a17d91f747a184f14341f4539f2c (127.15.147.130:44285)
I20260806 00:52:57.336740 11279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.130@KRBTEST.COM'} at 127.15.147.130:59807
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
Entry for principal kudu/127.15.147.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab.
I20260806 00:52:57.355221 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:33845
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:45215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:52:57.468242 11754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:57.468544 11754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:57.468585 11754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:57.472652 11754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:57.472702 11754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:57.472776 11754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:52:57.478149 11754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:45215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.131.keytab
--principal=kudu/127.15.147.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:33845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:57.479082 11754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:57.480028 11754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:57.490729 11761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:57.490698 11767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:57.490700 11763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:57.491083 11754 server_base.cc:1034] running on GCE node
I20260806 00:52:57.491484 11754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:57.492164 11754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:57.493326 11754 hybrid_clock.cc:648] HybridClock initialized: now 1785977577493312 us; error 25 us; skew 500 ppm
Aug 06 00:52:57 dist-test-slave-45dx krb5kdc[11212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977577, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:52:57.496236 11754 init.cc:377] Logged in from keytab as kudu/127.15.147.131@KRBTEST.COM (short username kudu)
I20260806 00:52:57.497210 11754 webserver.cc:460] Webserver started at http://127.15.147.131:45051/ using document root <none> and password file <none>
I20260806 00:52:57.497848 11754 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:57.497920 11754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:57.498165 11754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:57.500352 11754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "d67e3d6389704c4ba7f12e15b31fc7f5"
format_stamp: "Formatted at 2026-08-06 00:52:57 on dist-test-slave-45dx"
server_key: "baaa1a7ee4540e3dd55663d5cace81ea"
server_key_iv: "64b1c2c07121897e92cdc725c1e53c30"
server_key_version: "encryptionkey@0"
I20260806 00:52:57.500957 11754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "d67e3d6389704c4ba7f12e15b31fc7f5"
format_stamp: "Formatted at 2026-08-06 00:52:57 on dist-test-slave-45dx"
server_key: "baaa1a7ee4540e3dd55663d5cace81ea"
server_key_iv: "64b1c2c07121897e92cdc725c1e53c30"
server_key_version: "encryptionkey@0"
I20260806 00:52:57.504798 11754 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 00:52:57.507395 11777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:57.508749 11754 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:52:57.508846 11754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "d67e3d6389704c4ba7f12e15b31fc7f5"
format_stamp: "Formatted at 2026-08-06 00:52:57 on dist-test-slave-45dx"
server_key: "baaa1a7ee4540e3dd55663d5cace81ea"
server_key_iv: "64b1c2c07121897e92cdc725c1e53c30"
server_key_version: "encryptionkey@0"
I20260806 00:52:57.508939 11754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:57.528617 11754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:57.532140 11754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:57.532351 11754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:57.532972 11754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:57.533854 11754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:57.533912 11754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:57.533954 11754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:57.533969 11754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:57.543857 11754 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:42679
I20260806 00:52:57.543901 11930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:42679 every 8 connection(s)
I20260806 00:52:57.544929 11754 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:52:57.552367 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 11754
I20260806 00:52:57.552533 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:52:57.552886 15950 external_mini_cluster.cc:1428] Setting key 90803054ce7e2417ff7c49ffe0e4abc0
Aug 06 00:52:57 dist-test-slave-45dx krb5kdc[11212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977577, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.131@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:52:57.559055 11334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.546887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:43143 (local address 127.15.147.190:33845)
0806 00:52:57.547063 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:57.547068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:57.547884 (+ 816us) server_negotiation.cc:407] Connection header received
0806 00:52:57.549196 (+ 1312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:57.549199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:57.549248 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:57.549323 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:57.550485 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.550998 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.552178 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.552452 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.552594 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:57.555592 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:57.555595 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:57.555597 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:57.555623 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:57.557209 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:57.557693 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:57.557696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:57.557697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:57.557800 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:57.558094 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:57.558096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:57.558097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:57.558268 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:57.558341 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:57.558836 (+ 495us) server_negotiation.cc:299] Negotiation successful
0806 00:52:57.558938 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 00:52:57.559816 11938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.547220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:33845 (local address 127.15.147.131:43143)
0806 00:52:57.547698 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:57.547740 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:57.548979 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:57.549505 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:57.549511 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:57.549878 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:57.550358 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:57.550372 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.551111 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.551114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:57.552052 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:57.552065 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.552619 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.552623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:57.552873 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:57.553546 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:52:57.553586 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:57.555448 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:57.557324 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:57.557329 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:57.557340 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:57.557606 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:57.557881 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:52:57.557883 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:52:57.557884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:52:57.558015 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:52:57.558352 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:57.558357 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:57.558581 (+ 224us) client_negotiation.cc:769] Sending connection context
0806 00:52:57.558728 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 00:52:57.558967 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":125,"threads_started":1}
I20260806 00:52:57.560897 11932 heartbeater.cc:347] Connected to a master server at 127.15.147.190:33845
I20260806 00:52:57.561137 11932 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:57.561617 11932 heartbeater.cc:511] Master 127.15.147.190:33845 requested a full tablet report, sending...
I20260806 00:52:57.562659 11279 ts_manager.cc:194] Registered new tserver with Master: d67e3d6389704c4ba7f12e15b31fc7f5 (127.15.147.131:42679)
I20260806 00:52:57.563220 11279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.147.131@KRBTEST.COM'} at 127.15.147.131:43143
I20260806 00:52:57.566782 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:52:57.576393 11334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.568850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59696 (local address 127.15.147.190:33845)
0806 00:52:57.568963 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:57.568993 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:57.569262 (+ 269us) server_negotiation.cc:407] Connection header received
0806 00:52:57.569371 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:57.569374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:57.569421 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:57.569510 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:57.570217 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.570739 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.571548 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.571783 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.571901 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:57.573244 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:57.573247 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:57.573250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:52:57.573287 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:57.574634 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:57.575076 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:57.575079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:57.575081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:57.575161 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:52:57.575471 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:52:57.575475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:52:57.575476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:52:57.575688 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:57.575841 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:57.576093 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 00:52:57.576209 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 00:52:57.579169 11279 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:52:57.582163 11279 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:52:57.596446 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.591379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60280 (local address 127.15.147.130:44285)
0806 00:52:57.591781 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:57.591793 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:57.591814 (+ 21us) server_negotiation.cc:407] Connection header received
0806 00:52:57.591910 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:57.591919 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:57.592566 (+ 647us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:57.592714 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:57.592939 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.593826 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.595082 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.595702 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.595841 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:57.595977 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:57.596178 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 00:52:57.596273 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":176,"threads_started":1}
I20260806 00:52:57.596438 11950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.590387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:44285 (local address 127.0.0.1:60280)
0806 00:52:57.591065 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:57.591078 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:57.591172 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:57.592713 (+ 1541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:57.592716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:57.592735 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:57.592810 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:57.592815 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.593970 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.593974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:57.594924 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:57.594935 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.595839 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.595843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:57.595971 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:57.596017 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 00:52:57.596146 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 00:52:57.596263 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"spinlock_wait_cycles":4736,"thread_start_us":106,"threads_started":1}
I20260806 00:52:57.596797 11947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.590786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:42679 (local address 127.0.0.1:52982)
0806 00:52:57.591083 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:57.591097 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:57.591172 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:57.592079 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:57.592082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:57.592102 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:57.592217 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:57.592224 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.593389 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.593392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:57.595063 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:57.595075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.596424 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.596427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:57.596505 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:57.596532 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 00:52:57.596609 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 00:52:57.596666 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":145,"spinlock_wait_cycles":5120,"thread_start_us":117,"threads_started":1}
I20260806 00:52:57.597035 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.590162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:38805 (local address 127.0.0.1:41436)
0806 00:52:57.590747 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:57.590783 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:57.590887 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:57.591720 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:57.591726 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:57.591751 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:57.591881 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:57.591889 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.593508 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.593511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:57.595063 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:57.595075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.596219 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.596223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:57.596310 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:57.596360 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 00:52:57.596416 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 00:52:57.596486 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":151,"thread_start_us":78,"threads_started":1}
I20260806 00:52:57.597347 11951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.591256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52982 (local address 127.15.147.131:42679)
0806 00:52:57.591639 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:57.591652 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:57.591675 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:52:57.591748 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:57.591753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:57.591935 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:57.592096 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:57.592380 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.593251 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.595207 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.596078 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.596211 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:57.596361 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:57.597096 (+ 735us) server_negotiation.cc:299] Negotiation successful
0806 00:52:57.597209 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":156,"threads_started":1}
I20260806 00:52:57.597671 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.590332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41436 (local address 127.15.147.129:38805)
0806 00:52:57.591156 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:57.591176 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:57.591202 (+ 26us) server_negotiation.cc:407] Connection header received
0806 00:52:57.591282 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:57.591288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:57.591520 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:57.592044 (+ 524us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:57.592192 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.593366 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.595207 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.596078 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.597293 (+ 1215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:57.597431 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:57.597484 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 00:52:57.597569 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":113,"threads_started":1}
I20260806 00:52:57.599381 11648 tablet_service.cc:1467] Processing CreateTablet for tablet 7caef3739b244d069684ad9e695e1965 (DEFAULT_TABLE table=test-table [id=cfc17e378cec487aba39dad86f128691]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:57.599810 11457 tablet_service.cc:1467] Processing CreateTablet for tablet 7caef3739b244d069684ad9e695e1965 (DEFAULT_TABLE table=test-table [id=cfc17e378cec487aba39dad86f128691]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:57.600265 11648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7caef3739b244d069684ad9e695e1965. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:57.600629 11457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7caef3739b244d069684ad9e695e1965. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:57.603626 11847 tablet_service.cc:1467] Processing CreateTablet for tablet 7caef3739b244d069684ad9e695e1965 (DEFAULT_TABLE table=test-table [id=cfc17e378cec487aba39dad86f128691]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:57.604605 11847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7caef3739b244d069684ad9e695e1965. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:57.607512 11956 tablet_bootstrap.cc:492] T 7caef3739b244d069684ad9e695e1965 P 60fb4fb164ca4889921e8014f7ab9fbc: Bootstrap starting.
I20260806 00:52:57.608459 11954 tablet_bootstrap.cc:492] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c: Bootstrap starting.
I20260806 00:52:57.609982 11956 tablet_bootstrap.cc:654] T 7caef3739b244d069684ad9e695e1965 P 60fb4fb164ca4889921e8014f7ab9fbc: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:57.610658 11954 tablet_bootstrap.cc:654] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:57.611094 11957 tablet_bootstrap.cc:492] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5: Bootstrap starting.
I20260806 00:52:57.611481 11956 log.cc:824] T 7caef3739b244d069684ad9e695e1965 P 60fb4fb164ca4889921e8014f7ab9fbc: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:57.612198 11954 log.cc:824] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:57.613435 11957 tablet_bootstrap.cc:654] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:57.613888 11956 tablet_bootstrap.cc:492] T 7caef3739b244d069684ad9e695e1965 P 60fb4fb164ca4889921e8014f7ab9fbc: No bootstrap required, opened a new log
I20260806 00:52:57.614146 11956 ts_tablet_manager.cc:1397] T 7caef3739b244d069684ad9e695e1965 P 60fb4fb164ca4889921e8014f7ab9fbc: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260806 00:52:57.614382 11954 tablet_bootstrap.cc:492] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c: No bootstrap required, opened a new log
I20260806 00:52:57.614576 11954 ts_tablet_manager.cc:1397] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.003s
I20260806 00:52:57.614708 11957 log.cc:824] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:57.616717 11957 tablet_bootstrap.cc:492] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5: No bootstrap required, opened a new log
I20260806 00:52:57.616895 11957 ts_tablet_manager.cc:1397] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:52:57.617707 11956 raft_consensus.cc:348] T 7caef3739b244d069684ad9e695e1965 P 60fb4fb164ca4889921e8014f7ab9fbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60fb4fb164ca4889921e8014f7ab9fbc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38805 } } peers { permanent_uuid: "7de4a17d91f747a184f14341f4539f2c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44285 } } peers { permanent_uuid: "d67e3d6389704c4ba7f12e15b31fc7f5" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42679 } }
I20260806 00:52:57.617964 11956 raft_consensus.cc:374] T 7caef3739b244d069684ad9e695e1965 P 60fb4fb164ca4889921e8014f7ab9fbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:57.618021 11956 raft_consensus.cc:729] T 7caef3739b244d069684ad9e695e1965 P 60fb4fb164ca4889921e8014f7ab9fbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60fb4fb164ca4889921e8014f7ab9fbc, State: Initialized, Role: FOLLOWER
I20260806 00:52:57.618479 11956 consensus_queue.cc:260] T 7caef3739b244d069684ad9e695e1965 P 60fb4fb164ca4889921e8014f7ab9fbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60fb4fb164ca4889921e8014f7ab9fbc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38805 } } peers { permanent_uuid: "7de4a17d91f747a184f14341f4539f2c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44285 } } peers { permanent_uuid: "d67e3d6389704c4ba7f12e15b31fc7f5" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42679 } }
I20260806 00:52:57.618525 11954 raft_consensus.cc:348] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60fb4fb164ca4889921e8014f7ab9fbc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38805 } } peers { permanent_uuid: "7de4a17d91f747a184f14341f4539f2c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44285 } } peers { permanent_uuid: "d67e3d6389704c4ba7f12e15b31fc7f5" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42679 } }
I20260806 00:52:57.618773 11954 raft_consensus.cc:374] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:57.618847 11954 raft_consensus.cc:729] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7de4a17d91f747a184f14341f4539f2c, State: Initialized, Role: FOLLOWER
I20260806 00:52:57.619267 11555 heartbeater.cc:503] Master 127.15.147.190:33845 was elected leader, sending a full tablet report...
I20260806 00:52:57.619246 11954 consensus_queue.cc:260] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60fb4fb164ca4889921e8014f7ab9fbc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38805 } } peers { permanent_uuid: "7de4a17d91f747a184f14341f4539f2c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44285 } } peers { permanent_uuid: "d67e3d6389704c4ba7f12e15b31fc7f5" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42679 } }
I20260806 00:52:57.620235 11747 heartbeater.cc:503] Master 127.15.147.190:33845 was elected leader, sending a full tablet report...
I20260806 00:52:57.620268 11954 ts_tablet_manager.cc:1428] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260806 00:52:57.620847 11957 raft_consensus.cc:348] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60fb4fb164ca4889921e8014f7ab9fbc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38805 } } peers { permanent_uuid: "7de4a17d91f747a184f14341f4539f2c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44285 } } peers { permanent_uuid: "d67e3d6389704c4ba7f12e15b31fc7f5" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42679 } }
I20260806 00:52:57.621091 11957 raft_consensus.cc:374] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:57.621155 11957 raft_consensus.cc:729] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d67e3d6389704c4ba7f12e15b31fc7f5, State: Initialized, Role: FOLLOWER
I20260806 00:52:57.621608 11957 consensus_queue.cc:260] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60fb4fb164ca4889921e8014f7ab9fbc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38805 } } peers { permanent_uuid: "7de4a17d91f747a184f14341f4539f2c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44285 } } peers { permanent_uuid: "d67e3d6389704c4ba7f12e15b31fc7f5" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42679 } }
I20260806 00:52:57.622256 11932 heartbeater.cc:503] Master 127.15.147.190:33845 was elected leader, sending a full tablet report...
I20260806 00:52:57.622743 11956 ts_tablet_manager.cc:1428] T 7caef3739b244d069684ad9e695e1965 P 60fb4fb164ca4889921e8014f7ab9fbc: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260806 00:52:57.623502 11957 ts_tablet_manager.cc:1428] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 00:52:57.752872 11966 raft_consensus.cc:482] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:52:57.753134 11966 raft_consensus.cc:504] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60fb4fb164ca4889921e8014f7ab9fbc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38805 } } peers { permanent_uuid: "7de4a17d91f747a184f14341f4539f2c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44285 } } peers { permanent_uuid: "d67e3d6389704c4ba7f12e15b31fc7f5" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42679 } }
I20260806 00:52:57.754307 11966 leader_election.cc:288] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60fb4fb164ca4889921e8014f7ab9fbc (127.15.147.129:38805), d67e3d6389704c4ba7f12e15b31fc7f5 (127.15.147.131:42679)
I20260806 00:52:57.758638 11951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.754717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:42865 (local address 127.15.147.131:42679)
0806 00:52:57.754868 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:57.754874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:57.754907 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:52:57.755044 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:57.755047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:57.755094 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:57.755180 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:57.755598 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.756218 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.757564 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.758033 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.758158 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:57.758215 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:57.758423 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 00:52:57.758509 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:52:57.758641 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.754608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:42679 (local address 127.15.147.130:42865)
0806 00:52:57.754781 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:57.754799 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:57.754903 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:57.755228 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:57.755232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:57.755259 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:57.755471 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:57.755481 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.756353 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.756356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:57.757426 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:57.757436 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.758163 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.758167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:57.758272 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:57.758316 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 00:52:57.758382 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 00:52:57.758466 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":3}
I20260806 00:52:57.758687 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.754835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:50803 (local address 127.15.147.129:38805)
0806 00:52:57.754988 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:57.754992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:57.755220 (+ 228us) server_negotiation.cc:407] Connection header received
0806 00:52:57.755360 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:57.755364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:57.755420 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:57.755513 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:57.755845 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.756493 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.757628 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.758048 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.758182 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:57.758235 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:57.758496 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 00:52:57.758581 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 00:52:57.758663 11969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.754608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:38805 (local address 127.15.147.130:50803)
0806 00:52:57.755120 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:57.755135 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:57.755234 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:57.755552 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:57.755557 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:57.755578 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:57.755698 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:57.755704 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.756608 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.756612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:57.757456 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:57.757466 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.758195 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.758199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:57.758336 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:57.758355 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 00:52:57.758460 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 00:52:57.758523 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":195,"thread_start_us":123,"threads_started":1}
I20260806 00:52:57.759459 11872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7caef3739b244d069684ad9e695e1965" candidate_uuid: "7de4a17d91f747a184f14341f4539f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d67e3d6389704c4ba7f12e15b31fc7f5" is_pre_election: true
I20260806 00:52:57.759567 11486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7caef3739b244d069684ad9e695e1965" candidate_uuid: "7de4a17d91f747a184f14341f4539f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60fb4fb164ca4889921e8014f7ab9fbc" is_pre_election: true
I20260806 00:52:57.759804 11486 raft_consensus.cc:2454] T 7caef3739b244d069684ad9e695e1965 P 60fb4fb164ca4889921e8014f7ab9fbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7de4a17d91f747a184f14341f4539f2c in term 0.
I20260806 00:52:57.759796 11872 raft_consensus.cc:2454] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7de4a17d91f747a184f14341f4539f2c in term 0.
I20260806 00:52:57.760294 11582 leader_election.cc:302] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60fb4fb164ca4889921e8014f7ab9fbc, 7de4a17d91f747a184f14341f4539f2c; no voters:
I20260806 00:52:57.760632 11966 raft_consensus.cc:2781] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:52:57.760722 11966 raft_consensus.cc:482] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:52:57.760766 11966 raft_consensus.cc:3037] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:57.761838 11966 raft_consensus.cc:504] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60fb4fb164ca4889921e8014f7ab9fbc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38805 } } peers { permanent_uuid: "7de4a17d91f747a184f14341f4539f2c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44285 } } peers { permanent_uuid: "d67e3d6389704c4ba7f12e15b31fc7f5" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42679 } }
I20260806 00:52:57.762200 11966 leader_election.cc:288] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [CANDIDATE]: Term 1 election: Requested vote from peers 60fb4fb164ca4889921e8014f7ab9fbc (127.15.147.129:38805), d67e3d6389704c4ba7f12e15b31fc7f5 (127.15.147.131:42679)
I20260806 00:52:57.762555 11486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7caef3739b244d069684ad9e695e1965" candidate_uuid: "7de4a17d91f747a184f14341f4539f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60fb4fb164ca4889921e8014f7ab9fbc"
I20260806 00:52:57.762642 11872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7caef3739b244d069684ad9e695e1965" candidate_uuid: "7de4a17d91f747a184f14341f4539f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d67e3d6389704c4ba7f12e15b31fc7f5"
I20260806 00:52:57.762705 11486 raft_consensus.cc:3037] T 7caef3739b244d069684ad9e695e1965 P 60fb4fb164ca4889921e8014f7ab9fbc [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:57.762755 11872 raft_consensus.cc:3037] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:57.764138 11486 raft_consensus.cc:2454] T 7caef3739b244d069684ad9e695e1965 P 60fb4fb164ca4889921e8014f7ab9fbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7de4a17d91f747a184f14341f4539f2c in term 1.
I20260806 00:52:57.764138 11872 raft_consensus.cc:2454] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7de4a17d91f747a184f14341f4539f2c in term 1.
I20260806 00:52:57.764493 11582 leader_election.cc:302] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60fb4fb164ca4889921e8014f7ab9fbc, 7de4a17d91f747a184f14341f4539f2c; no voters:
I20260806 00:52:57.764768 11966 raft_consensus.cc:2781] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:52:57.765031 11966 raft_consensus.cc:686] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [term 1 LEADER]: Becoming Leader. State: Replica: 7de4a17d91f747a184f14341f4539f2c, State: Running, Role: LEADER
I20260806 00:52:57.765368 11966 consensus_queue.cc:237] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60fb4fb164ca4889921e8014f7ab9fbc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38805 } } peers { permanent_uuid: "7de4a17d91f747a184f14341f4539f2c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44285 } } peers { permanent_uuid: "d67e3d6389704c4ba7f12e15b31fc7f5" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42679 } }
I20260806 00:52:57.768896 11275 catalog_manager.cc:5495] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7de4a17d91f747a184f14341f4539f2c (127.15.147.130). New cstate: current_term: 1 leader_uuid: "7de4a17d91f747a184f14341f4539f2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60fb4fb164ca4889921e8014f7ab9fbc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7de4a17d91f747a184f14341f4539f2c" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 44285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d67e3d6389704c4ba7f12e15b31fc7f5" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42679 } health_report { overall_health: UNKNOWN } } }
I20260806 00:52:57.787860 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.783975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60292 (local address 127.15.147.130:44285)
0806 00:52:57.784103 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:57.784107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:57.784120 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:52:57.784268 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:57.784270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:57.784383 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:57.784497 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:52:57.784762 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.785270 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.786456 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:57.786666 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:57.786743 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:57.787071 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:52:57.787430 (+ 359us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:52:57.787515 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:57.787647 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 00:52:57.787698 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:52:57.794015 11872 raft_consensus.cc:1261] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5 [term 1 FOLLOWER]: Refusing update from remote peer 7de4a17d91f747a184f14341f4539f2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:52:57.794015 11486 raft_consensus.cc:1261] T 7caef3739b244d069684ad9e695e1965 P 60fb4fb164ca4889921e8014f7ab9fbc [term 1 FOLLOWER]: Refusing update from remote peer 7de4a17d91f747a184f14341f4539f2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:52:57.794785 11970 consensus_queue.cc:1035] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d67e3d6389704c4ba7f12e15b31fc7f5" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:52:57.795002 11966 consensus_queue.cc:1035] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "60fb4fb164ca4889921e8014f7ab9fbc" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 00:52:57.795941 11934 tablet.cc:2369] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:52:57.803408 11974 mvcc.cc:205] Tried to move back new op lower bound from 7315364158636441600 to 7315364158529482752. Current Snapshot: MvccSnapshot[applied={T|T < 7315364158636441600}]
I20260806 00:52:57.804939 11976 mvcc.cc:205] Tried to move back new op lower bound from 7315364158636441600 to 7315364158529482752. Current Snapshot: MvccSnapshot[applied={T|T < 7315364158636441600}]
I20260806 00:52:57.811597 11279 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:52:57.811786 11279 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:52:57.814348 11279 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6e75b5c9140472b9fe4fffe4e03f63d: Sending DeleteTablet for 3 replicas of tablet 7caef3739b244d069684ad9e695e1965
I20260806 00:52:57.815184 11847 tablet_service.cc:1514] Processing DeleteTablet for tablet 7caef3739b244d069684ad9e695e1965 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:57 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:52982
I20260806 00:52:57.815176 11648 tablet_service.cc:1514] Processing DeleteTablet for tablet 7caef3739b244d069684ad9e695e1965 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:57 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:60280
I20260806 00:52:57.815352 11457 tablet_service.cc:1514] Processing DeleteTablet for tablet 7caef3739b244d069684ad9e695e1965 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:57 UTC) from {username='kudu', principal='kudu/127.15.147.190@KRBTEST.COM'} at 127.0.0.1:41436
I20260806 00:52:57.815871 11985 tablet_replica.cc:330] T 7caef3739b244d069684ad9e695e1965 P 60fb4fb164ca4889921e8014f7ab9fbc: stopping tablet replica
I20260806 00:52:57.815894 11984 tablet_replica.cc:330] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5: stopping tablet replica
I20260806 00:52:57.816200 11985 raft_consensus.cc:2229] T 7caef3739b244d069684ad9e695e1965 P 60fb4fb164ca4889921e8014f7ab9fbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:52:57.816200 11984 raft_consensus.cc:2229] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:52:57.816569 11984 raft_consensus.cc:2258] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:57.816571 11985 raft_consensus.cc:2258] T 7caef3739b244d069684ad9e695e1965 P 60fb4fb164ca4889921e8014f7ab9fbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:57.816745 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 11372
I20260806 00:52:57.817083 11982 tablet_replica.cc:330] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c: stopping tablet replica
I20260806 00:52:57.817313 11982 raft_consensus.cc:2229] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:52:57.817641 11982 raft_consensus.cc:2258] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:57.820703 11984 ts_tablet_manager.cc:1905] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:52:57.820748 11982 ts_tablet_manager.cc:1905] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:52:57.823909 11982 ts_tablet_manager.cc:1918] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:52:57.824013 11982 log.cc:1196] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/7caef3739b244d069684ad9e695e1965
I20260806 00:52:57.824220 11984 ts_tablet_manager.cc:1918] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:52:57.824349 11984 log.cc:1196] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/7caef3739b244d069684ad9e695e1965
I20260806 00:52:57.824391 11982 ts_tablet_manager.cc:1939] T 7caef3739b244d069684ad9e695e1965 P 7de4a17d91f747a184f14341f4539f2c: Deleting consensus metadata
I20260806 00:52:57.824709 11984 ts_tablet_manager.cc:1939] T 7caef3739b244d069684ad9e695e1965 P d67e3d6389704c4ba7f12e15b31fc7f5: Deleting consensus metadata
I20260806 00:52:57.825680 11259 catalog_manager.cc:4841] TS 7de4a17d91f747a184f14341f4539f2c (127.15.147.130:44285): tablet 7caef3739b244d069684ad9e695e1965 (table test-table [id=cfc17e378cec487aba39dad86f128691]) successfully deleted
I20260806 00:52:57.826026 11260 catalog_manager.cc:4841] TS d67e3d6389704c4ba7f12e15b31fc7f5 (127.15.147.131:42679): tablet 7caef3739b244d069684ad9e695e1965 (table test-table [id=cfc17e378cec487aba39dad86f128691]) successfully deleted
W20260806 00:52:57.833158 11258 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.147.129:38805 (error 108)
I20260806 00:52:57.833341 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 11563
I20260806 00:52:57.837307 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:57.836129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:38805 (local address 127.0.0.1:41440)
0806 00:52:57.837105 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:57.837194 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:38805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":860}
W20260806 00:52:57.837462 11258 catalog_manager.cc:4584] TS 60fb4fb164ca4889921e8014f7ab9fbc (127.15.147.129:38805): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7caef3739b244d069684ad9e695e1965: Network error: Client connection negotiation failed: client connection to 127.15.147.129:38805: connect: Connection refused (error 111)
I20260806 00:52:57.844846 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 11754
I20260806 00:52:57.860181 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 11235
2026-08-06T00:52:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2604 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T00:52:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:52:57Z Disabled control of system clock
I20260806 00:52:57.895126 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:44301
--webserver_interface=127.15.147.190
--webserver_port=0
--builtin_ntp_servers=127.15.147.148:41757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:44301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 00:52:58.007946 11991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:58.008203 11991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:58.008256 11991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:58.012200 11991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:52:58.012257 11991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:58.012272 11991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:58.012288 11991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:52:58.012358 11991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:52:58.017145 11991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:41757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:44301
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:44301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:58.018216 11991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:58.019621 11991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:52:58.028244 11991 server_base.cc:1034] running on GCE node
W20260806 00:52:58.028257 11997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:58.028196 11996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:58.028174 12000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:58.028929 11991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:58.030066 11991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:58.031255 11991 hybrid_clock.cc:648] HybridClock initialized: now 1785977578031235 us; error 45 us; skew 500 ppm
I20260806 00:52:58.033093 11991 webserver.cc:460] Webserver started at http://127.15.147.190:36777/ using document root <none> and password file <none>
I20260806 00:52:58.033674 11991 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:58.033766 11991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:58.034004 11991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:58.036038 11991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "ae91171153ee4e478758f30282e3d8e3"
format_stamp: "Formatted at 2026-08-06 00:52:58 on dist-test-slave-45dx"
server_key: "aa625bf08d288c7b29fc142112342601"
server_key_iv: "bbb72039c590cb2ab86e372ac6e9904e"
server_key_version: "encryptionkey@0"
I20260806 00:52:58.036652 11991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "ae91171153ee4e478758f30282e3d8e3"
format_stamp: "Formatted at 2026-08-06 00:52:58 on dist-test-slave-45dx"
server_key: "aa625bf08d288c7b29fc142112342601"
server_key_iv: "bbb72039c590cb2ab86e372ac6e9904e"
server_key_version: "encryptionkey@0"
I20260806 00:52:58.040236 11991 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260806 00:52:58.042629 12009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:58.043783 11991 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 00:52:58.043931 11991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "ae91171153ee4e478758f30282e3d8e3"
format_stamp: "Formatted at 2026-08-06 00:52:58 on dist-test-slave-45dx"
server_key: "aa625bf08d288c7b29fc142112342601"
server_key_iv: "bbb72039c590cb2ab86e372ac6e9904e"
server_key_version: "encryptionkey@0"
I20260806 00:52:58.044044 11991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:58.067301 11991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:58.068004 11991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:58.068210 11991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:58.075495 11991 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:44301
I20260806 00:52:58.075502 12085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:44301 every 8 connection(s)
I20260806 00:52:58.076496 11991 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:52:58.079496 12088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:58.081390 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 11991
I20260806 00:52:58.081540 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:52:58.081848 15950 external_mini_cluster.cc:1428] Setting key 804871daa702a65103d63e0b381e0c2b
I20260806 00:52:58.085402 12088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3: Bootstrap starting.
I20260806 00:52:58.087831 12088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:58.088747 12088 log.cc:824] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:58.090759 12088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3: No bootstrap required, opened a new log
I20260806 00:52:58.091042 12095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:58.083311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57642 (local address 127.15.147.190:44301)
0806 00:52:58.083896 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:58.083910 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:58.083942 (+ 32us) server_negotiation.cc:407] Connection header received
0806 00:52:58.084637 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:58.084654 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:58.085011 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:58.085257 (+ 246us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:58.085653 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.086581 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.087666 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.088131 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.088259 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:58.089230 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:58.089237 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:58.089252 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:52:58.089731 (+ 479us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:58.089784 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:52:58.089804 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:58.089880 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:58.090058 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 00:52:58.090333 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":137,"threads_started":1}
I20260806 00:52:58.094269 12088 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae91171153ee4e478758f30282e3d8e3" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44301 } }
I20260806 00:52:58.094501 12088 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:58.094576 12088 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae91171153ee4e478758f30282e3d8e3, State: Initialized, Role: FOLLOWER
I20260806 00:52:58.095017 12088 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae91171153ee4e478758f30282e3d8e3" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44301 } }
I20260806 00:52:58.095161 12088 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:52:58.095243 12088 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:52:58.095366 12088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:58.096421 12088 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae91171153ee4e478758f30282e3d8e3" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44301 } }
I20260806 00:52:58.096758 12088 leader_election.cc:302] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae91171153ee4e478758f30282e3d8e3; no voters:
I20260806 00:52:58.097087 12088 leader_election.cc:288] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:52:58.097240 12098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:52:58.097493 12098 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3 [term 1 LEADER]: Becoming Leader. State: Replica: ae91171153ee4e478758f30282e3d8e3, State: Running, Role: LEADER
I20260806 00:52:58.097797 12098 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae91171153ee4e478758f30282e3d8e3" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44301 } }
I20260806 00:52:58.098284 12088 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:52:58.099563 12099 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae91171153ee4e478758f30282e3d8e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae91171153ee4e478758f30282e3d8e3" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44301 } } }
I20260806 00:52:58.099715 12099 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3 [sys.catalog]: This master's current role is: LEADER
I20260806 00:52:58.100083 12113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:52:58.100054 12100 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae91171153ee4e478758f30282e3d8e3. Latest consensus state: current_term: 1 leader_uuid: "ae91171153ee4e478758f30282e3d8e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae91171153ee4e478758f30282e3d8e3" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44301 } } }
I20260806 00:52:58.100215 12100 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3 [sys.catalog]: This master's current role is: LEADER
I20260806 00:52:58.102818 12113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:52:58.108119 12113 catalog_manager.cc:1347] Generated new cluster ID: fc1689cb414348259d0d6343df8cc391
I20260806 00:52:58.108193 12113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:52:58.130357 12113 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:52:58.131582 12113 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:52:58.138209 12113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3: Generated new TSK 0
I20260806 00:52:58.138978 12113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:52:58.147498 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.129:0
--local_ip_for_outbound_sockets=127.15.147.129
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:44301
--builtin_ntp_servers=127.15.147.148:41757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 00:52:58.262395 12129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:58.262797 12129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:58.262866 12129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:58.269681 12129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:58.269778 12129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:58.269860 12129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.129
I20260806 00:52:58.274729 12129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:41757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.147.129
--webserver_port=0
--tserver_master_addrs=127.15.147.190:44301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.129
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:58.275820 12129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:58.277282 12129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:58.287385 12139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:58.287379 12136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:58.287355 12134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:58.287451 12129 server_base.cc:1034] running on GCE node
I20260806 00:52:58.288112 12129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:58.288810 12129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:58.290002 12129 hybrid_clock.cc:648] HybridClock initialized: now 1785977578289978 us; error 48 us; skew 500 ppm
I20260806 00:52:58.291913 12129 webserver.cc:460] Webserver started at http://127.15.147.129:39735/ using document root <none> and password file <none>
I20260806 00:52:58.292577 12129 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:58.292672 12129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:58.292917 12129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:58.294987 12129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-0/data/instance:
uuid: "2bdf10b79de547e0834a4d9f8161908b"
format_stamp: "Formatted at 2026-08-06 00:52:58 on dist-test-slave-45dx"
server_key: "c4caed74b519f34fe5b26fdb123547a5"
server_key_iv: "97431df127ac61b968e43d982c768f5a"
server_key_version: "encryptionkey@0"
I20260806 00:52:58.295585 12129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance:
uuid: "2bdf10b79de547e0834a4d9f8161908b"
format_stamp: "Formatted at 2026-08-06 00:52:58 on dist-test-slave-45dx"
server_key: "c4caed74b519f34fe5b26fdb123547a5"
server_key_iv: "97431df127ac61b968e43d982c768f5a"
server_key_version: "encryptionkey@0"
I20260806 00:52:58.299314 12129 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:52:58.301926 12146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:58.303144 12129 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:52:58.303282 12129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-0/wal
uuid: "2bdf10b79de547e0834a4d9f8161908b"
format_stamp: "Formatted at 2026-08-06 00:52:58 on dist-test-slave-45dx"
server_key: "c4caed74b519f34fe5b26fdb123547a5"
server_key_iv: "97431df127ac61b968e43d982c768f5a"
server_key_version: "encryptionkey@0"
I20260806 00:52:58.303398 12129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:58.313472 12129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:58.314136 12129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:58.314332 12129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:58.314932 12129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:58.315897 12129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:58.315969 12129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:58.316042 12129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:58.316095 12129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:58.325937 12129 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.129:38433
I20260806 00:52:58.325961 12315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.129:38433 every 8 connection(s)
I20260806 00:52:58.326839 12129 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-0/data/info.pb
I20260806 00:52:58.334977 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 12129
I20260806 00:52:58.335124 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-0/wal/instance
I20260806 00:52:58.335515 15950 external_mini_cluster.cc:1428] Setting key eee0c75e9f33d965cf9845f1381f6d8f
I20260806 00:52:58.335892 12095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:58.328703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.129:34891 (local address 127.15.147.190:44301)
0806 00:52:58.328842 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:58.328845 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:58.329904 (+ 1059us) server_negotiation.cc:407] Connection header received
0806 00:52:58.331274 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:58.331277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:58.331322 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:58.331400 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:58.332250 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.332829 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.333935 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.334145 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.334216 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:58.335364 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:58.335368 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:58.335376 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:52:58.335389 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:58.335409 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:52:58.335417 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:58.335476 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:58.335717 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 00:52:58.335774 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 00:52:58.336946 12318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:58.329109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:44301 (local address 127.15.147.129:34891)
0806 00:52:58.329755 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:58.329796 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:58.331086 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:58.331541 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:58.331547 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:58.331627 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:58.332126 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:58.332139 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.332953 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.332955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:58.333819 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:58.333828 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.334272 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.334275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:58.334410 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:58.335057 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:52:58.335060 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:58.335122 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:52:58.335203 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:58.335507 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:58.335517 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:58.335525 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 00:52:58.335708 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 00:52:58.336037 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":112,"threads_started":1}
I20260806 00:52:58.337709 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.130:0
--local_ip_for_outbound_sockets=127.15.147.130
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:44301
--builtin_ntp_servers=127.15.147.148:41757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 00:52:58.339082 12316 heartbeater.cc:347] Connected to a master server at 127.15.147.190:44301
I20260806 00:52:58.339354 12316 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:58.339965 12316 heartbeater.cc:511] Master 127.15.147.190:44301 requested a full tablet report, sending...
I20260806 00:52:58.341643 12032 ts_manager.cc:194] Registered new tserver with Master: 2bdf10b79de547e0834a4d9f8161908b (127.15.147.129:38433)
I20260806 00:52:58.342810 12032 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.147.129:34891
W20260806 00:52:58.451609 12319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:58.451905 12319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:58.451973 12319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:58.455973 12319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:58.456054 12319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:58.456146 12319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.130
I20260806 00:52:58.460783 12319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:41757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.147.130
--webserver_port=0
--tserver_master_addrs=127.15.147.190:44301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.130
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:58.462054 12319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:58.463522 12319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:58.473259 12328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:58.473285 12326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:58.473353 12319 server_base.cc:1034] running on GCE node
W20260806 00:52:58.473261 12325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:58.473881 12319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:58.474500 12319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:58.475685 12319 hybrid_clock.cc:648] HybridClock initialized: now 1785977578475654 us; error 43 us; skew 500 ppm
I20260806 00:52:58.477545 12319 webserver.cc:460] Webserver started at http://127.15.147.130:43359/ using document root <none> and password file <none>
I20260806 00:52:58.478098 12319 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:58.478192 12319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:58.478421 12319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:58.480424 12319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-1/data/instance:
uuid: "a897c8ee918840ae89f455c3b41ca867"
format_stamp: "Formatted at 2026-08-06 00:52:58 on dist-test-slave-45dx"
server_key: "9aa56b4a81b9c5fe5c1584aef5ed5a28"
server_key_iv: "b14ef1fc958df439907202dd2a3c9ce8"
server_key_version: "encryptionkey@0"
I20260806 00:52:58.481035 12319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance:
uuid: "a897c8ee918840ae89f455c3b41ca867"
format_stamp: "Formatted at 2026-08-06 00:52:58 on dist-test-slave-45dx"
server_key: "9aa56b4a81b9c5fe5c1584aef5ed5a28"
server_key_iv: "b14ef1fc958df439907202dd2a3c9ce8"
server_key_version: "encryptionkey@0"
I20260806 00:52:58.484634 12319 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:52:58.486955 12335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:58.488097 12319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:52:58.488227 12319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-1/wal
uuid: "a897c8ee918840ae89f455c3b41ca867"
format_stamp: "Formatted at 2026-08-06 00:52:58 on dist-test-slave-45dx"
server_key: "9aa56b4a81b9c5fe5c1584aef5ed5a28"
server_key_iv: "b14ef1fc958df439907202dd2a3c9ce8"
server_key_version: "encryptionkey@0"
I20260806 00:52:58.488396 12319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:58.504556 12319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:58.505204 12319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:58.505411 12319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:58.506001 12319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:58.506945 12319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:58.507015 12319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:58.507089 12319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:58.507131 12319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:58.516062 12319 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.130:39009
I20260806 00:52:58.516088 12512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.130:39009 every 8 connection(s)
I20260806 00:52:58.516968 12319 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-1/data/info.pb
I20260806 00:52:58.525746 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 12319
I20260806 00:52:58.525904 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-1/wal/instance
I20260806 00:52:58.526284 15950 external_mini_cluster.cc:1428] Setting key b08f4160ab93efd4763fae84dfc77002
I20260806 00:52:58.526501 12095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:58.518797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:34151 (local address 127.15.147.190:44301)
0806 00:52:58.518955 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:58.518958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:58.519961 (+ 1003us) server_negotiation.cc:407] Connection header received
0806 00:52:58.521463 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:58.521466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:58.521523 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:58.521634 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:58.522660 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.523202 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.524452 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.524671 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.524786 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:58.525952 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:58.525958 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:58.525962 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:52:58.525979 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:58.526004 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:52:58.526022 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:58.526088 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:58.526264 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 00:52:58.526315 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:52:58.527391 12515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:58.519182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:44301 (local address 127.15.147.130:34151)
0806 00:52:58.519813 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:58.519853 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:58.521248 (+ 1395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:58.521843 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:58.521850 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:58.521931 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:58.522511 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:58.522526 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.523349 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.523352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:58.524243 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:58.524253 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.524830 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.524834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:58.525001 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:58.525625 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:52:58.525627 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:58.525681 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:52:58.525764 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:58.526102 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:58.526113 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:58.526120 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 00:52:58.526255 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 00:52:58.526558 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":101,"threads_started":1}
I20260806 00:52:58.528491 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.147.131:0
--local_ip_for_outbound_sockets=127.15.147.131
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:44301
--builtin_ntp_servers=127.15.147.148:41757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 00:52:58.528529 12513 heartbeater.cc:347] Connected to a master server at 127.15.147.190:44301
I20260806 00:52:58.528756 12513 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:58.529327 12513 heartbeater.cc:511] Master 127.15.147.190:44301 requested a full tablet report, sending...
I20260806 00:52:58.530372 12032 ts_manager.cc:194] Registered new tserver with Master: a897c8ee918840ae89f455c3b41ca867 (127.15.147.130:39009)
I20260806 00:52:58.530927 12032 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.147.130:34151
W20260806 00:52:58.641687 12516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:52:58.642004 12516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:52:58.642082 12516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:52:58.645984 12516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:52:58.646055 12516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:52:58.646138 12516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.147.131
I20260806 00:52:58.650977 12516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:41757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.147.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.147.131
--webserver_port=0
--tserver_master_addrs=127.15.147.190:44301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.147.131
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:52:58.652030 12516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:52:58.653420 12516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:52:58.662986 12527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:58.662971 12523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:52:58.663002 12521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:52:58.663329 12516 server_base.cc:1034] running on GCE node
I20260806 00:52:58.663760 12516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:52:58.664374 12516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:52:58.665560 12516 hybrid_clock.cc:648] HybridClock initialized: now 1785977578665529 us; error 42 us; skew 500 ppm
I20260806 00:52:58.667337 12516 webserver.cc:460] Webserver started at http://127.15.147.131:35389/ using document root <none> and password file <none>
I20260806 00:52:58.667901 12516 fs_manager.cc:359] Metadata directory not provided
I20260806 00:52:58.667963 12516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:52:58.668203 12516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:52:58.670303 12516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-2/data/instance:
uuid: "f4ac298bc47144b3bb3bea0a950b62a2"
format_stamp: "Formatted at 2026-08-06 00:52:58 on dist-test-slave-45dx"
server_key: "5afab083263431b9b2316ae1b664413c"
server_key_iv: "580b823bf601a07bd88ab12d9f1434ce"
server_key_version: "encryptionkey@0"
I20260806 00:52:58.670881 12516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance:
uuid: "f4ac298bc47144b3bb3bea0a950b62a2"
format_stamp: "Formatted at 2026-08-06 00:52:58 on dist-test-slave-45dx"
server_key: "5afab083263431b9b2316ae1b664413c"
server_key_iv: "580b823bf601a07bd88ab12d9f1434ce"
server_key_version: "encryptionkey@0"
I20260806 00:52:58.674479 12516 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:52:58.677004 12534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:58.678207 12516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260806 00:52:58.678344 12516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-2/wal
uuid: "f4ac298bc47144b3bb3bea0a950b62a2"
format_stamp: "Formatted at 2026-08-06 00:52:58 on dist-test-slave-45dx"
server_key: "5afab083263431b9b2316ae1b664413c"
server_key_iv: "580b823bf601a07bd88ab12d9f1434ce"
server_key_version: "encryptionkey@0"
I20260806 00:52:58.678457 12516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:52:58.710297 12516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:52:58.711069 12516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:52:58.711282 12516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:52:58.711962 12516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 00:52:58.712939 12516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 00:52:58.712990 12516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:58.713057 12516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 00:52:58.713099 12516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 00:52:58.722141 12516 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.131:42511
I20260806 00:52:58.722167 12691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.131:42511 every 8 connection(s)
I20260806 00:52:58.723155 12516 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-2/data/info.pb
I20260806 00:52:58.726460 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 12516
I20260806 00:52:58.726590 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-2/wal/instance
I20260806 00:52:58.726902 15950 external_mini_cluster.cc:1428] Setting key 70d09aa90c1e1b93981b40cb9c4e6b16
I20260806 00:52:58.732566 12095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:58.725093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.131:38433 (local address 127.15.147.190:44301)
0806 00:52:58.725223 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:58.725227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:58.726275 (+ 1048us) server_negotiation.cc:407] Connection header received
0806 00:52:58.727924 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:58.727926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:58.727995 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:58.728093 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:58.729011 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.729534 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.730670 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.730897 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.731007 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:58.732053 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:58.732057 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:58.732060 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:52:58.732078 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:58.732098 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:52:58.732106 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:58.732184 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:58.732356 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 00:52:58.732427 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:52:58.733337 12697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:58.725464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:44301 (local address 127.15.147.131:38433)
0806 00:52:58.726100 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:58.726141 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:58.727700 (+ 1559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:58.728272 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:58.728282 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:58.728403 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:52:58.728859 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:58.728871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.729684 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.729687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:58.730543 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:58.730553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.731050 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.731054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:58.731214 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:58.731792 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 00:52:58.731794 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:52:58.731834 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 00:52:58.731904 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:52:58.732186 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:52:58.732199 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:52:58.732205 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 00:52:58.732357 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 00:52:58.732619 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":115,"threads_started":1}
I20260806 00:52:58.734349 12693 heartbeater.cc:347] Connected to a master server at 127.15.147.190:44301
I20260806 00:52:58.734579 12693 heartbeater.cc:464] Registering TS with master...
I20260806 00:52:58.735087 12693 heartbeater.cc:511] Master 127.15.147.190:44301 requested a full tablet report, sending...
I20260806 00:52:58.736071 12032 ts_manager.cc:194] Registered new tserver with Master: f4ac298bc47144b3bb3bea0a950b62a2 (127.15.147.131:42511)
I20260806 00:52:58.736701 12032 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.147.131:38433
I20260806 00:52:58.740918 15950 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 00:52:58.746364 12095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:58.742882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57656 (local address 127.15.147.190:44301)
0806 00:52:58.743050 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:58.743053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:58.743205 (+ 152us) server_negotiation.cc:407] Connection header received
0806 00:52:58.743280 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:58.743282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:58.743337 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:58.743414 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:52:58.743625 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.744097 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.744975 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.745180 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.745264 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:58.746057 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:52:58.746060 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:52:58.746062 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:52:58.746072 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:52:58.746086 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:52:58.746093 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:52:58.746148 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:58.746223 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 00:52:58.746266 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 00:52:58.749244 12032 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57656:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 00:52:58.752211 12032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 00:52:58.766175 12708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:58.760439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33410 (local address 127.15.147.129:38433)
0806 00:52:58.760952 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:58.760966 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:58.761210 (+ 244us) server_negotiation.cc:407] Connection header received
0806 00:52:58.761438 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:58.761444 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:58.761636 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:58.761775 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:58.762379 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.763584 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.764934 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.765472 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.765613 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:58.765786 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:58.765893 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 00:52:58.766002 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":137,"threads_started":1}
I20260806 00:52:58.766330 12706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:58.760094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.130:39009 (local address 127.0.0.1:40034)
0806 00:52:58.760613 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:58.760663 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:58.760788 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:58.761963 (+ 1175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:58.761970 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:58.761995 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:58.762268 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:58.762274 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.763724 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.763728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:58.764868 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:58.764876 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.765912 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.765916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:58.766012 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:58.766034 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 00:52:58.766080 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 00:52:58.766144 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":114,"thread_start_us":75,"threads_started":1}
I20260806 00:52:58.766166 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:58.760826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:42511 (local address 127.0.0.1:58708)
0806 00:52:58.761134 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:58.761150 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:58.761238 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:58.762013 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:58.762017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:58.762036 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:58.762263 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:58.762272 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.763284 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.763290 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:58.764586 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:58.764598 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.765660 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.765664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:58.765773 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:58.765809 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 00:52:58.765862 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 00:52:58.766033 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":102,"thread_start_us":105,"threads_started":1}
I20260806 00:52:58.766543 12711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:58.761050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58708 (local address 127.15.147.131:42511)
0806 00:52:58.761498 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:58.761511 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:58.761534 (+ 23us) server_negotiation.cc:407] Connection header received
0806 00:52:58.761607 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:58.761611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:58.761841 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:58.762031 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:58.762409 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.763166 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.764725 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.765511 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.766056 (+ 545us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:58.766226 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:58.766275 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 00:52:58.766378 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":96,"threads_started":1}
I20260806 00:52:58.766875 12707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:58.760232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:38433 (local address 127.0.0.1:33410)
0806 00:52:58.761103 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:58.761118 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:58.761282 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:58.761987 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:58.761989 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:58.762001 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:58.762239 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:58.762247 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.763744 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.763747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:58.764804 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:58.764813 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.765603 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.765607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:58.765712 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:58.765748 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 00:52:58.765865 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 00:52:58.766753 (+ 888us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"mutex_wait_us":96,"thread_start_us":116,"threads_started":1}
I20260806 00:52:58.767241 12712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:58.761040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40034 (local address 127.15.147.130:39009)
0806 00:52:58.761505 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:58.761517 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:58.761537 (+ 20us) server_negotiation.cc:407] Connection header received
0806 00:52:58.761602 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:58.761606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:58.761753 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:58.762219 (+ 466us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:58.762371 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.763585 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.765000 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:58.765765 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:58.766803 (+ 1038us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:58.766963 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:58.767008 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 00:52:58.767090 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":131,"threads_started":1}
I20260806 00:52:58.769515 12211 tablet_service.cc:1467] Processing CreateTablet for tablet d8c8b923ca96479baefc0d9c9861b5be (DEFAULT_TABLE table=test-table [id=f67f2d8d31ed4096867ad2253eb55ffa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:58.769644 12408 tablet_service.cc:1467] Processing CreateTablet for tablet d8c8b923ca96479baefc0d9c9861b5be (DEFAULT_TABLE table=test-table [id=f67f2d8d31ed4096867ad2253eb55ffa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:58.769644 12605 tablet_service.cc:1467] Processing CreateTablet for tablet d8c8b923ca96479baefc0d9c9861b5be (DEFAULT_TABLE table=test-table [id=f67f2d8d31ed4096867ad2253eb55ffa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 00:52:58.770542 12211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8c8b923ca96479baefc0d9c9861b5be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:58.770642 12605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8c8b923ca96479baefc0d9c9861b5be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:58.770642 12408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8c8b923ca96479baefc0d9c9861b5be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:52:58.776824 12713 tablet_bootstrap.cc:492] T d8c8b923ca96479baefc0d9c9861b5be P 2bdf10b79de547e0834a4d9f8161908b: Bootstrap starting.
I20260806 00:52:58.778396 12716 tablet_bootstrap.cc:492] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2: Bootstrap starting.
I20260806 00:52:58.778396 12717 tablet_bootstrap.cc:492] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867: Bootstrap starting.
I20260806 00:52:58.779062 12713 tablet_bootstrap.cc:654] T d8c8b923ca96479baefc0d9c9861b5be P 2bdf10b79de547e0834a4d9f8161908b: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:58.779794 12713 log.cc:824] T d8c8b923ca96479baefc0d9c9861b5be P 2bdf10b79de547e0834a4d9f8161908b: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:58.780493 12716 tablet_bootstrap.cc:654] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:58.780493 12717 tablet_bootstrap.cc:654] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867: Neither blocks nor log segments found. Creating new log.
I20260806 00:52:58.781349 12716 log.cc:824] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:58.781358 12717 log.cc:824] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867: Log is configured to *not* fsync() on all Append() calls
I20260806 00:52:58.782068 12713 tablet_bootstrap.cc:492] T d8c8b923ca96479baefc0d9c9861b5be P 2bdf10b79de547e0834a4d9f8161908b: No bootstrap required, opened a new log
I20260806 00:52:58.782307 12713 ts_tablet_manager.cc:1397] T d8c8b923ca96479baefc0d9c9861b5be P 2bdf10b79de547e0834a4d9f8161908b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 00:52:58.783172 12717 tablet_bootstrap.cc:492] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867: No bootstrap required, opened a new log
I20260806 00:52:58.783361 12717 ts_tablet_manager.cc:1397] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 00:52:58.783492 12716 tablet_bootstrap.cc:492] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2: No bootstrap required, opened a new log
I20260806 00:52:58.783665 12716 ts_tablet_manager.cc:1397] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260806 00:52:58.786665 12717 raft_consensus.cc:348] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a897c8ee918840ae89f455c3b41ca867" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39009 } } peers { permanent_uuid: "2bdf10b79de547e0834a4d9f8161908b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38433 } } peers { permanent_uuid: "f4ac298bc47144b3bb3bea0a950b62a2" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42511 } }
I20260806 00:52:58.786895 12717 raft_consensus.cc:374] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:58.786767 12713 raft_consensus.cc:348] T d8c8b923ca96479baefc0d9c9861b5be P 2bdf10b79de547e0834a4d9f8161908b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a897c8ee918840ae89f455c3b41ca867" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39009 } } peers { permanent_uuid: "2bdf10b79de547e0834a4d9f8161908b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38433 } } peers { permanent_uuid: "f4ac298bc47144b3bb3bea0a950b62a2" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42511 } }
I20260806 00:52:58.786962 12717 raft_consensus.cc:729] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a897c8ee918840ae89f455c3b41ca867, State: Initialized, Role: FOLLOWER
I20260806 00:52:58.787019 12713 raft_consensus.cc:374] T d8c8b923ca96479baefc0d9c9861b5be P 2bdf10b79de547e0834a4d9f8161908b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:58.787083 12713 raft_consensus.cc:729] T d8c8b923ca96479baefc0d9c9861b5be P 2bdf10b79de547e0834a4d9f8161908b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bdf10b79de547e0834a4d9f8161908b, State: Initialized, Role: FOLLOWER
I20260806 00:52:58.787459 12717 consensus_queue.cc:260] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a897c8ee918840ae89f455c3b41ca867" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39009 } } peers { permanent_uuid: "2bdf10b79de547e0834a4d9f8161908b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38433 } } peers { permanent_uuid: "f4ac298bc47144b3bb3bea0a950b62a2" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42511 } }
I20260806 00:52:58.787513 12713 consensus_queue.cc:260] T d8c8b923ca96479baefc0d9c9861b5be P 2bdf10b79de547e0834a4d9f8161908b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a897c8ee918840ae89f455c3b41ca867" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39009 } } peers { permanent_uuid: "2bdf10b79de547e0834a4d9f8161908b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38433 } } peers { permanent_uuid: "f4ac298bc47144b3bb3bea0a950b62a2" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42511 } }
I20260806 00:52:58.788090 12513 heartbeater.cc:503] Master 127.15.147.190:44301 was elected leader, sending a full tablet report...
I20260806 00:52:58.787817 12716 raft_consensus.cc:348] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a897c8ee918840ae89f455c3b41ca867" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39009 } } peers { permanent_uuid: "2bdf10b79de547e0834a4d9f8161908b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38433 } } peers { permanent_uuid: "f4ac298bc47144b3bb3bea0a950b62a2" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42511 } }
I20260806 00:52:58.788245 12716 raft_consensus.cc:374] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:52:58.788335 12716 raft_consensus.cc:729] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4ac298bc47144b3bb3bea0a950b62a2, State: Initialized, Role: FOLLOWER
I20260806 00:52:58.788538 12713 ts_tablet_manager.cc:1428] T d8c8b923ca96479baefc0d9c9861b5be P 2bdf10b79de547e0834a4d9f8161908b: Time spent starting tablet: real 0.006s user 0.001s sys 0.006s
I20260806 00:52:58.788779 12716 consensus_queue.cc:260] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a897c8ee918840ae89f455c3b41ca867" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39009 } } peers { permanent_uuid: "2bdf10b79de547e0834a4d9f8161908b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38433 } } peers { permanent_uuid: "f4ac298bc47144b3bb3bea0a950b62a2" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42511 } }
I20260806 00:52:58.789139 12316 heartbeater.cc:503] Master 127.15.147.190:44301 was elected leader, sending a full tablet report...
I20260806 00:52:58.789582 12716 ts_tablet_manager.cc:1428] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 00:52:58.790140 12693 heartbeater.cc:503] Master 127.15.147.190:44301 was elected leader, sending a full tablet report...
I20260806 00:52:58.791771 12717 ts_tablet_manager.cc:1428] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
W20260806 00:52:58.828064 12317 tablet.cc:2369] T d8c8b923ca96479baefc0d9c9861b5be P 2bdf10b79de547e0834a4d9f8161908b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:52:58.974267 12694 tablet.cc:2369] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 00:52:59.022763 12514 tablet.cc:2369] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 00:52:59.110496 12723 raft_consensus.cc:482] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 00:52:59.110708 12723 raft_consensus.cc:504] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a897c8ee918840ae89f455c3b41ca867" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39009 } } peers { permanent_uuid: "2bdf10b79de547e0834a4d9f8161908b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38433 } } peers { permanent_uuid: "f4ac298bc47144b3bb3bea0a950b62a2" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42511 } }
I20260806 00:52:59.112010 12723 leader_election.cc:288] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2bdf10b79de547e0834a4d9f8161908b (127.15.147.129:38433), f4ac298bc47144b3bb3bea0a950b62a2 (127.15.147.131:42511)
I20260806 00:52:59.116043 12729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:59.112232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:38433 (local address 127.15.147.130:51219)
0806 00:52:59.112621 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:59.112636 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:59.112748 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:59.113002 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:59.113005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:59.113023 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:59.113163 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:59.113170 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:59.114005 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:59.114009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:59.114876 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:59.114887 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:59.115585 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:59.115591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:59.115707 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:59.115749 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 00:52:59.115815 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 00:52:59.115869 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":8,"thread_start_us":144,"threads_started":1}
I20260806 00:52:59.116086 12708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:59.112394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:51219 (local address 127.15.147.129:38433)
0806 00:52:59.112544 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:59.112548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:59.112733 (+ 185us) server_negotiation.cc:407] Connection header received
0806 00:52:59.112841 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:59.112845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:59.112896 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:59.112976 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:59.113308 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:59.113875 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:59.115011 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:59.115448 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:59.115546 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:59.115621 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:59.115859 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 00:52:59.115946 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 00:52:59.116226 12711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:59.112484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.147.130:55421 (local address 127.15.147.131:42511)
0806 00:52:59.112613 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:59.112617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:59.112873 (+ 256us) server_negotiation.cc:407] Connection header received
0806 00:52:59.113009 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:59.113012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:59.113060 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:59.113141 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 00:52:59.113466 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:59.114031 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:59.115082 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:59.115519 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:59.115641 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:59.115689 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:59.116004 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 00:52:59.116099 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:52:59.116590 12731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:59.112362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.131:42511 (local address 127.15.147.130:55421)
0806 00:52:59.112739 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:59.112754 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:52:59.112865 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:52:59.113165 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:52:59.113168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:52:59.113183 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 00:52:59.113348 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:59.113354 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:59.114144 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:59.114147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:59.114953 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:52:59.114967 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:52:59.115661 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:52:59.115665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:52:59.115842 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:59.115867 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 00:52:59.115962 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 00:52:59.116028 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":53,"thread_start_us":84,"threads_started":1}
I20260806 00:52:59.116745 12241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c8b923ca96479baefc0d9c9861b5be" candidate_uuid: "a897c8ee918840ae89f455c3b41ca867" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bdf10b79de547e0834a4d9f8161908b" is_pre_election: true
I20260806 00:52:59.117126 12241 raft_consensus.cc:2454] T d8c8b923ca96479baefc0d9c9861b5be P 2bdf10b79de547e0834a4d9f8161908b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a897c8ee918840ae89f455c3b41ca867 in term 0.
I20260806 00:52:59.117292 12629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c8b923ca96479baefc0d9c9861b5be" candidate_uuid: "a897c8ee918840ae89f455c3b41ca867" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4ac298bc47144b3bb3bea0a950b62a2" is_pre_election: true
I20260806 00:52:59.117574 12629 raft_consensus.cc:2454] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a897c8ee918840ae89f455c3b41ca867 in term 0.
I20260806 00:52:59.117638 12342 leader_election.cc:302] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bdf10b79de547e0834a4d9f8161908b, a897c8ee918840ae89f455c3b41ca867; no voters:
I20260806 00:52:59.117880 12723 raft_consensus.cc:2781] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 00:52:59.117972 12723 raft_consensus.cc:482] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 00:52:59.118042 12723 raft_consensus.cc:3037] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:59.119292 12723 raft_consensus.cc:504] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a897c8ee918840ae89f455c3b41ca867" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39009 } } peers { permanent_uuid: "2bdf10b79de547e0834a4d9f8161908b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38433 } } peers { permanent_uuid: "f4ac298bc47144b3bb3bea0a950b62a2" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42511 } }
I20260806 00:52:59.119666 12723 leader_election.cc:288] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [CANDIDATE]: Term 1 election: Requested vote from peers 2bdf10b79de547e0834a4d9f8161908b (127.15.147.129:38433), f4ac298bc47144b3bb3bea0a950b62a2 (127.15.147.131:42511)
I20260806 00:52:59.119975 12241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c8b923ca96479baefc0d9c9861b5be" candidate_uuid: "a897c8ee918840ae89f455c3b41ca867" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bdf10b79de547e0834a4d9f8161908b"
I20260806 00:52:59.120081 12629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8c8b923ca96479baefc0d9c9861b5be" candidate_uuid: "a897c8ee918840ae89f455c3b41ca867" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4ac298bc47144b3bb3bea0a950b62a2"
I20260806 00:52:59.120086 12241 raft_consensus.cc:3037] T d8c8b923ca96479baefc0d9c9861b5be P 2bdf10b79de547e0834a4d9f8161908b [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:59.120225 12629 raft_consensus.cc:3037] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:52:59.121173 12241 raft_consensus.cc:2454] T d8c8b923ca96479baefc0d9c9861b5be P 2bdf10b79de547e0834a4d9f8161908b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a897c8ee918840ae89f455c3b41ca867 in term 1.
I20260806 00:52:59.121337 12629 raft_consensus.cc:2454] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a897c8ee918840ae89f455c3b41ca867 in term 1.
I20260806 00:52:59.121512 12342 leader_election.cc:302] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bdf10b79de547e0834a4d9f8161908b, a897c8ee918840ae89f455c3b41ca867; no voters:
I20260806 00:52:59.121714 12723 raft_consensus.cc:2781] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:52:59.121950 12723 raft_consensus.cc:686] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [term 1 LEADER]: Becoming Leader. State: Replica: a897c8ee918840ae89f455c3b41ca867, State: Running, Role: LEADER
I20260806 00:52:59.122269 12723 consensus_queue.cc:237] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a897c8ee918840ae89f455c3b41ca867" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39009 } } peers { permanent_uuid: "2bdf10b79de547e0834a4d9f8161908b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38433 } } peers { permanent_uuid: "f4ac298bc47144b3bb3bea0a950b62a2" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42511 } }
I20260806 00:52:59.125578 12030 catalog_manager.cc:5495] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 reported cstate change: term changed from 0 to 1, leader changed from <none> to a897c8ee918840ae89f455c3b41ca867 (127.15.147.130). New cstate: current_term: 1 leader_uuid: "a897c8ee918840ae89f455c3b41ca867" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a897c8ee918840ae89f455c3b41ca867" member_type: VOTER last_known_addr { host: "127.15.147.130" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bdf10b79de547e0834a4d9f8161908b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4ac298bc47144b3bb3bea0a950b62a2" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42511 } health_report { overall_health: UNKNOWN } } }
I20260806 00:52:59.215996 12712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:59.212582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40038 (local address 127.15.147.130:39009)
0806 00:52:59.212728 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:52:59.212732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:52:59.212746 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:52:59.212825 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:52:59.212827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:52:59.212903 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:52:59.212998 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:52:59.213290 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:59.213762 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:59.214616 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:52:59.214843 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:52:59.214924 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:52:59.215210 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:52:59.215634 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:52:59.215723 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 00:52:59.215816 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 00:52:59.215866 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:52:59.221853 12629 raft_consensus.cc:1261] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2 [term 1 FOLLOWER]: Refusing update from remote peer a897c8ee918840ae89f455c3b41ca867: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:52:59.221853 12241 raft_consensus.cc:1261] T d8c8b923ca96479baefc0d9c9861b5be P 2bdf10b79de547e0834a4d9f8161908b [term 1 FOLLOWER]: Refusing update from remote peer a897c8ee918840ae89f455c3b41ca867: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:52:59.222577 12723 consensus_queue.cc:1035] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bdf10b79de547e0834a4d9f8161908b" member_type: VOTER last_known_addr { host: "127.15.147.129" port: 38433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:52:59.222785 12732 consensus_queue.cc:1035] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4ac298bc47144b3bb3bea0a950b62a2" member_type: VOTER last_known_addr { host: "127.15.147.131" port: 42511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:52:59.239334 12030 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:52:59.239545 12030 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 00:52:59.242194 12030 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae91171153ee4e478758f30282e3d8e3: Sending DeleteTablet for 3 replicas of tablet d8c8b923ca96479baefc0d9c9861b5be
I20260806 00:52:59.243108 12211 tablet_service.cc:1514] Processing DeleteTablet for tablet d8c8b923ca96479baefc0d9c9861b5be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:59 UTC) from {username='slave'} at 127.0.0.1:33410
I20260806 00:52:59.243201 12605 tablet_service.cc:1514] Processing DeleteTablet for tablet d8c8b923ca96479baefc0d9c9861b5be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:59 UTC) from {username='slave'} at 127.0.0.1:58708
I20260806 00:52:59.243242 12408 tablet_service.cc:1514] Processing DeleteTablet for tablet d8c8b923ca96479baefc0d9c9861b5be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 00:52:59 UTC) from {username='slave'} at 127.0.0.1:40034
I20260806 00:52:59.243662 12751 tablet_replica.cc:330] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2: stopping tablet replica
I20260806 00:52:59.244017 12751 raft_consensus.cc:2229] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:52:59.244104 12754 tablet_replica.cc:330] T d8c8b923ca96479baefc0d9c9861b5be P 2bdf10b79de547e0834a4d9f8161908b: stopping tablet replica
I20260806 00:52:59.244410 12751 raft_consensus.cc:2258] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:59.244462 12754 raft_consensus.cc:2229] T d8c8b923ca96479baefc0d9c9861b5be P 2bdf10b79de547e0834a4d9f8161908b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 00:52:59.244738 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 12129
I20260806 00:52:59.244774 12754 raft_consensus.cc:2258] T d8c8b923ca96479baefc0d9c9861b5be P 2bdf10b79de547e0834a4d9f8161908b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:59.253571 12757 tablet_replica.cc:330] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867: stopping tablet replica
I20260806 00:52:59.254007 12757 raft_consensus.cc:2229] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [term 1 LEADER]: Raft consensus shutting down.
I20260806 00:52:59.254400 12757 raft_consensus.cc:2258] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 00:52:59.255952 12757 ts_tablet_manager.cc:1905] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:52:59.258315 12751 ts_tablet_manager.cc:1905] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 00:52:59.258891 12757 ts_tablet_manager.cc:1918] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:52:59.258998 12757 log.cc:1196] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-1/wal/wals/d8c8b923ca96479baefc0d9c9861b5be
I20260806 00:52:59.259328 12757 ts_tablet_manager.cc:1939] T d8c8b923ca96479baefc0d9c9861b5be P a897c8ee918840ae89f455c3b41ca867: Deleting consensus metadata
I20260806 00:52:59.260679 12010 catalog_manager.cc:4841] TS a897c8ee918840ae89f455c3b41ca867 (127.15.147.130:39009): tablet d8c8b923ca96479baefc0d9c9861b5be (table test-table [id=f67f2d8d31ed4096867ad2253eb55ffa]) successfully deleted
W20260806 00:52:59.261059 12014 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.147.129:38433 (error 108)
I20260806 00:52:59.262017 12751 ts_tablet_manager.cc:1918] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 00:52:59.262159 12707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:52:59.261827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.129:38433 (local address 127.0.0.1:33422)
0806 00:52:59.261989 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 00:52:59.262064 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.129:38433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 00:52:59.262277 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 12319
I20260806 00:52:59.262295 12751 log.cc:1196] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2: Deleting WAL directory at /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785977317010501-15950-0/minicluster-data/ts-2/wal/wals/d8c8b923ca96479baefc0d9c9861b5be
I20260806 00:52:59.262653 12751 ts_tablet_manager.cc:1939] T d8c8b923ca96479baefc0d9c9861b5be P f4ac298bc47144b3bb3bea0a950b62a2: Deleting consensus metadata
I20260806 00:52:59.264691 12010 catalog_manager.cc:4841] TS f4ac298bc47144b3bb3bea0a950b62a2 (127.15.147.131:42511): tablet d8c8b923ca96479baefc0d9c9861b5be (table test-table [id=f67f2d8d31ed4096867ad2253eb55ffa]) successfully deleted
W20260806 00:52:59.267488 12014 catalog_manager.cc:4584] TS 2bdf10b79de547e0834a4d9f8161908b (127.15.147.129:38433): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d8c8b923ca96479baefc0d9c9861b5be: Network error: Client connection negotiation failed: client connection to 127.15.147.129:38433: connect: Connection refused (error 111)
I20260806 00:52:59.275259 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 12516
I20260806 00:52:59.287391 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 11991
2026-08-06T00:52:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1430 ms)
[----------] 6 tests from EncryptionPolicyTest (11744 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:52:59 dist-test-slave-45dx krb5kdc[12761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:52:59 dist-test-slave-45dx krb5kdc[12761](info): set up 2 sockets
Aug 06 00:52:59 dist-test-slave-45dx krb5kdc[12761](info): commencing operation
krb5kdc: starting...
W20260806 00:53:00.509608 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:00 dist-test-slave-45dx krb5kdc[12761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:53:00.647365 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42049
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:00.759915 12778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:00.760167 12778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:00.760245 12778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:00.764185 12778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:00.764240 12778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:00.764258 12778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:00.764335 12778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:00.764364 12778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:00.769423 12778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42049
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:00.770462 12778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:00.771497 12778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:00.780328 12784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:00.780313 12786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:00.780318 12783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:00.780591 12778 server_base.cc:1034] running on GCE node
I20260806 00:53:00.781229 12778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:00.782320 12778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:00.783520 12778 hybrid_clock.cc:648] HybridClock initialized: now 1785977580783502 us; error 30 us; skew 500 ppm
Aug 06 00:53:00 dist-test-slave-45dx krb5kdc[12761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977580, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:00.786396 12778 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:53:00.787288 12778 webserver.cc:460] Webserver started at http://127.0.0.1:39803/ using document root <none> and password file <none>
I20260806 00:53:00.787854 12778 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:00.787951 12778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:00.788183 12778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:00.790243 12778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "06829e09d2bd4b17bfb817a714df36f1"
format_stamp: "Formatted at 2026-08-06 00:53:00 on dist-test-slave-45dx"
server_key: "9da769b836c1829e671adafd1489477a"
server_key_iv: "14c7ada919608c0ccd8172d1a2ed0703"
server_key_version: "encryptionkey@0"
I20260806 00:53:00.790827 12778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "06829e09d2bd4b17bfb817a714df36f1"
format_stamp: "Formatted at 2026-08-06 00:53:00 on dist-test-slave-45dx"
server_key: "9da769b836c1829e671adafd1489477a"
server_key_iv: "14c7ada919608c0ccd8172d1a2ed0703"
server_key_version: "encryptionkey@0"
I20260806 00:53:00.794337 12778 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:53:00.796824 12799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:00.798053 12778 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:53:00.798192 12778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "06829e09d2bd4b17bfb817a714df36f1"
format_stamp: "Formatted at 2026-08-06 00:53:00 on dist-test-slave-45dx"
server_key: "9da769b836c1829e671adafd1489477a"
server_key_iv: "14c7ada919608c0ccd8172d1a2ed0703"
server_key_version: "encryptionkey@0"
I20260806 00:53:00.798301 12778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:00.826568 12778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:00.829635 12778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:00.829844 12778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:00.836616 12778 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42049
I20260806 00:53:00.836644 12872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42049 every 8 connection(s)
I20260806 00:53:00.837555 12778 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:00.840193 12873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:00.845291 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 12778
I20260806 00:53:00.845427 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:00.845768 15950 external_mini_cluster.cc:1428] Setting key b78d43921ceba8b44d30f0d73ea36d50
I20260806 00:53:00.845937 12873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1: Bootstrap starting.
I20260806 00:53:00.848430 12873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:00.849184 12873 log.cc:824] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:00.850991 12873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1: No bootstrap required, opened a new log
Aug 06 00:53:00 dist-test-slave-45dx krb5kdc[12761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 00:53:00.855012 12873 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06829e09d2bd4b17bfb817a714df36f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42049 } }
I20260806 00:53:00.855269 12873 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:00.855347 12873 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06829e09d2bd4b17bfb817a714df36f1, State: Initialized, Role: FOLLOWER
I20260806 00:53:00.855877 12873 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06829e09d2bd4b17bfb817a714df36f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42049 } }
I20260806 00:53:00.856011 12873 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:00.856112 12873 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:00.856230 12873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:00.857108 12873 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06829e09d2bd4b17bfb817a714df36f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42049 } }
I20260806 00:53:00.857496 12873 leader_election.cc:302] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06829e09d2bd4b17bfb817a714df36f1; no voters:
I20260806 00:53:00.857874 12873 leader_election.cc:288] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:00.857985 12884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:00.858289 12884 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1 [term 1 LEADER]: Becoming Leader. State: Replica: 06829e09d2bd4b17bfb817a714df36f1, State: Running, Role: LEADER
I20260806 00:53:00.858677 12884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06829e09d2bd4b17bfb817a714df36f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42049 } }
I20260806 00:53:00.859119 12873 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:00.860474 12886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06829e09d2bd4b17bfb817a714df36f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06829e09d2bd4b17bfb817a714df36f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42049 } } }
I20260806 00:53:00.860594 12886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:00.860971 12887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06829e09d2bd4b17bfb817a714df36f1. Latest consensus state: current_term: 1 leader_uuid: "06829e09d2bd4b17bfb817a714df36f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06829e09d2bd4b17bfb817a714df36f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42049 } } }
I20260806 00:53:00.861042 12896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:00.861079 12887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:00.861045 12879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:00.847182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37843 (local address 127.0.0.1:42049)
0806 00:53:00.847607 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:00.847623 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:00.847657 (+ 34us) server_negotiation.cc:407] Connection header received
0806 00:53:00.848265 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:00.848281 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:00.848677 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:00.849012 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:00.849857 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:00.850789 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:00.851694 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:00.852144 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:00.852358 (+ 214us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:00.855161 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:00.855165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:00.855177 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:00.855204 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:00.857108 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:00.857792 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:00.857802 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:00.857811 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:00.857981 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:00.858466 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:00.858470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:00.858472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:00.858973 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:00.859157 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:00.859696 (+ 539us) server_negotiation.cc:299] Negotiation successful
0806 00:53:00.859965 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":134,"threads_started":1}
I20260806 00:53:00.864177 12896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:00.865777 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:00.870244 12896 catalog_manager.cc:1347] Generated new cluster ID: 60fd241100184947a56b073e9c8c6935
I20260806 00:53:00.870332 12896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:00.880916 12879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:00.870939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51835 (local address 127.0.0.1:42049)
0806 00:53:00.871038 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:00.871042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:00.871059 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:53:00.872553 (+ 1494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:00.872559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:00.872616 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:00.872708 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:00.873426 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:00.874274 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:00.875380 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:00.875742 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:00.875850 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:00.877488 (+ 1638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:00.877492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:00.877495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:00.877521 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:00.879151 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:00.879656 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:00.879661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:00.879664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:00.879748 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:00.880067 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:00.880071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:00.880072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:00.880269 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:00.880441 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:00.880673 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 00:53:00.880766 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 00:53:00.895298 12896 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:00.896201 12896 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:00.905682 12896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06829e09d2bd4b17bfb817a714df36f1: Generated new TSK 0
I20260806 00:53:00.906422 12896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:00.910136 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 12778
W20260806 00:53:00.920894 12759 connection.cc:535] client connection to 127.0.0.1:42049 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42049): Connection reset by peer (error 104)
2026-08-06T00:53:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1619 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:00 dist-test-slave-45dx krb5kdc[12913](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:00 dist-test-slave-45dx krb5kdc[12913](info): set up 2 sockets
Aug 06 00:53:00 dist-test-slave-45dx krb5kdc[12913](info): commencing operation
krb5kdc: starting...
W20260806 00:53:02.119153 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:02 dist-test-slave-45dx krb5kdc[12913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:53:02.255734 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44827
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:02.374948 12932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:02.375240 12932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:02.375284 12932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:02.379436 12932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:02.379496 12932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:02.379515 12932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:02.379532 12932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:02.379547 12932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:02.385141 12932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44827
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:02.386253 12932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:02.387306 12932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:02.395895 12941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:02.395925 12944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:02.395918 12940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:02.396355 12932 server_base.cc:1034] running on GCE node
I20260806 00:53:02.396874 12932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:02.397898 12932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:02.399101 12932 hybrid_clock.cc:648] HybridClock initialized: now 1785977582399072 us; error 39 us; skew 500 ppm
Aug 06 00:53:02 dist-test-slave-45dx krb5kdc[12913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:02.405444 12932 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:53:02.406301 12932 webserver.cc:460] Webserver started at http://127.0.0.1:40531/ using document root <none> and password file <none>
I20260806 00:53:02.406821 12932 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:02.406880 12932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:02.407059 12932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:02.409238 12932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "2ce87279ca3a4f208d9de3c17f247938"
format_stamp: "Formatted at 2026-08-06 00:53:02 on dist-test-slave-45dx"
server_key: "c0327758f63e52ba610708f895369c81"
server_key_iv: "7f90b8ef2f5510f3d92777fc9e397359"
server_key_version: "encryptionkey@0"
I20260806 00:53:02.409781 12932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "2ce87279ca3a4f208d9de3c17f247938"
format_stamp: "Formatted at 2026-08-06 00:53:02 on dist-test-slave-45dx"
server_key: "c0327758f63e52ba610708f895369c81"
server_key_iv: "7f90b8ef2f5510f3d92777fc9e397359"
server_key_version: "encryptionkey@0"
I20260806 00:53:02.413600 12932 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 00:53:02.416086 12952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:02.417454 12932 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:53:02.417630 12932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "2ce87279ca3a4f208d9de3c17f247938"
format_stamp: "Formatted at 2026-08-06 00:53:02 on dist-test-slave-45dx"
server_key: "c0327758f63e52ba610708f895369c81"
server_key_iv: "7f90b8ef2f5510f3d92777fc9e397359"
server_key_version: "encryptionkey@0"
I20260806 00:53:02.417742 12932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:02.440599 12932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:02.443751 12932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:02.443986 12932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:02.452869 12932 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44827
I20260806 00:53:02.452904 13024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44827 every 8 connection(s)
I20260806 00:53:02.454334 12932 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:02.458365 13025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:02.464239 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 12932
I20260806 00:53:02.464399 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:02.464730 15950 external_mini_cluster.cc:1428] Setting key ea185d72dc1478904b2d22d2bf1cb6ab
I20260806 00:53:02.466750 13025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938: Bootstrap starting.
I20260806 00:53:02.469702 13025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:02.470646 13025 log.cc:824] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:02.473384 13025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938: No bootstrap required, opened a new log
I20260806 00:53:02.478104 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:02.466753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43061 (local address 127.0.0.1:44827)
0806 00:53:02.467302 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:02.467326 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:02.467374 (+ 48us) server_negotiation.cc:407] Connection header received
0806 00:53:02.468641 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:02.468675 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:02.469329 (+ 654us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:02.469955 (+ 626us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:02.470525 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:02.472174 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:02.473585 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:02.474337 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:02.474544 (+ 207us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:02.475599 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:02.475606 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:02.475624 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:02.476177 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:02.476260 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:02.476292 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:02.476463 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:02.476840 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 00:53:02.477015 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":149,"threads_started":1}
I20260806 00:53:02.478605 13025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ce87279ca3a4f208d9de3c17f247938" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44827 } }
I20260806 00:53:02.478904 13025 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:02.478976 13025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ce87279ca3a4f208d9de3c17f247938, State: Initialized, Role: FOLLOWER
I20260806 00:53:02.479467 13025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ce87279ca3a4f208d9de3c17f247938" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44827 } }
I20260806 00:53:02.479607 13025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:02.479650 13025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:02.479720 13025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:02.481068 13025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ce87279ca3a4f208d9de3c17f247938" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44827 } }
I20260806 00:53:02.481410 13025 leader_election.cc:302] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ce87279ca3a4f208d9de3c17f247938; no voters:
I20260806 00:53:02.482730 13025 leader_election.cc:288] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:02.482807 13030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:02.484090 13030 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938 [term 1 LEADER]: Becoming Leader. State: Replica: 2ce87279ca3a4f208d9de3c17f247938, State: Running, Role: LEADER
I20260806 00:53:02.484460 13030 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ce87279ca3a4f208d9de3c17f247938" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44827 } }
I20260806 00:53:02.484869 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:02.485138 13025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:02.485731 13030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ce87279ca3a4f208d9de3c17f247938" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ce87279ca3a4f208d9de3c17f247938" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44827 } } }
I20260806 00:53:02.485857 13030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:02.486233 13032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ce87279ca3a4f208d9de3c17f247938. Latest consensus state: current_term: 1 leader_uuid: "2ce87279ca3a4f208d9de3c17f247938" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ce87279ca3a4f208d9de3c17f247938" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44827 } } }
I20260806 00:53:02.486351 13032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938 [sys.catalog]: This master's current role is: LEADER
W20260806 00:53:02.490296 13058 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:53:02.490377 13058 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:53:02.490437 13043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:02.491233 13043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:02.492676 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:02.487235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56371 (local address 127.0.0.1:44827)
0806 00:53:02.487369 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:02.487376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:02.488101 (+ 725us) server_negotiation.cc:407] Connection header received
0806 00:53:02.488203 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:02.488206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:02.488252 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:02.488395 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:02.488737 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:02.489507 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:02.490682 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:02.491034 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:02.491150 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:02.492144 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:02.492148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:02.492151 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:02.492164 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:02.492194 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:02.492208 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:02.492311 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:02.492497 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 00:53:02.492535 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 00:53:02.498229 13043 catalog_manager.cc:1347] Generated new cluster ID: c5891c981a3a4ed0944760d6974ee3a0
I20260806 00:53:02.498358 13043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:02.511902 13043 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:02.513182 13043 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:02.523932 13043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ce87279ca3a4f208d9de3c17f247938: Generated new TSK 0
I20260806 00:53:02.524734 13043 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:02.534926 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 12932
W20260806 00:53:02.546391 12911 connection.cc:535] client connection to 127.0.0.1:44827 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44827): Connection reset by peer (error 104)
2026-08-06T00:53:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1625 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:02 dist-test-slave-45dx krb5kdc[13068](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:02 dist-test-slave-45dx krb5kdc[13068](info): set up 2 sockets
Aug 06 00:53:02 dist-test-slave-45dx krb5kdc[13068](info): commencing operation
krb5kdc: starting...
W20260806 00:53:03.751394 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:03 dist-test-slave-45dx krb5kdc[13068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:53:03.889912 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38215
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:04.004061 13085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:04.004390 13085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:04.004472 13085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:04.008520 13085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:04.008605 13085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:04.008643 13085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:04.008682 13085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:04.008719 13085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:04.013931 13085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38215
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:04.015043 13085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:04.016089 13085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:04.024834 13091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:04.024895 13092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:04.024915 13095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:04.025233 13085 server_base.cc:1034] running on GCE node
I20260806 00:53:04.025920 13085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:04.027063 13085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:04.028270 13085 hybrid_clock.cc:648] HybridClock initialized: now 1785977584028233 us; error 45 us; skew 500 ppm
Aug 06 00:53:04 dist-test-slave-45dx krb5kdc[13068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977584, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:04.042119 13085 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:53:04.043025 13085 webserver.cc:460] Webserver started at http://127.0.0.1:39781/ using document root <none> and password file <none>
I20260806 00:53:04.043565 13085 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:04.043625 13085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:04.043808 13085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:04.045869 13085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "d0477c1c737247059170156450233079"
format_stamp: "Formatted at 2026-08-06 00:53:04 on dist-test-slave-45dx"
server_key: "c0daf8688b1c0077e288a114f57ad376"
server_key_iv: "8802259088de8559bd808319c66e741a"
server_key_version: "encryptionkey@0"
I20260806 00:53:04.046478 13085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "d0477c1c737247059170156450233079"
format_stamp: "Formatted at 2026-08-06 00:53:04 on dist-test-slave-45dx"
server_key: "c0daf8688b1c0077e288a114f57ad376"
server_key_iv: "8802259088de8559bd808319c66e741a"
server_key_version: "encryptionkey@0"
I20260806 00:53:04.050101 13085 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:53:04.052444 13106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:04.053611 13085 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:53:04.053740 13085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "d0477c1c737247059170156450233079"
format_stamp: "Formatted at 2026-08-06 00:53:04 on dist-test-slave-45dx"
server_key: "c0daf8688b1c0077e288a114f57ad376"
server_key_iv: "8802259088de8559bd808319c66e741a"
server_key_version: "encryptionkey@0"
I20260806 00:53:04.053838 13085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:04.058385 13085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:04.061249 13085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:04.061460 13085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:04.068759 13085 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38215
I20260806 00:53:04.068830 13175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38215 every 8 connection(s)
I20260806 00:53:04.069708 13085 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:04.072535 13178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:04.076839 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 13085
I20260806 00:53:04.076998 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:04.077316 15950 external_mini_cluster.cc:1428] Setting key eaf0d242a1362a5dc8a28b3edf50f95c
I20260806 00:53:04.078140 13178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079: Bootstrap starting.
I20260806 00:53:04.080515 13178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:04.081413 13178 log.cc:824] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:04.082647 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:04.078744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34677 (local address 127.0.0.1:38215)
0806 00:53:04.079177 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:04.079191 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:04.079226 (+ 35us) server_negotiation.cc:407] Connection header received
0806 00:53:04.079839 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:04.079855 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:04.080123 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:04.080481 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:04.081310 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:04.081315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:04.081325 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:04.081335 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:04.081390 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:04.081408 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:04.081540 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:04.081635 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 00:53:04.081837 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":132,"threads_started":1}
I20260806 00:53:04.083556 13178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079: No bootstrap required, opened a new log
I20260806 00:53:04.085371 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:04.087431 13178 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0477c1c737247059170156450233079" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } }
I20260806 00:53:04.087750 13178 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:04.087823 13178 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0477c1c737247059170156450233079, State: Initialized, Role: FOLLOWER
I20260806 00:53:04.088325 13178 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0477c1c737247059170156450233079" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } }
I20260806 00:53:04.088495 13178 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:04.088578 13178 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:04.088683 13178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:04.089285 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:04.087454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37435 (local address 127.0.0.1:38215)
0806 00:53:04.087600 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:04.087603 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:04.087814 (+ 211us) server_negotiation.cc:407] Connection header received
0806 00:53:04.087992 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:04.087994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:04.088023 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:04.088124 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:04.088878 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:04.088881 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:04.088882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:04.088892 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:04.088914 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:04.088924 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:04.088988 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:04.089102 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 00:53:04.089146 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 00:53:04.089887 13178 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0477c1c737247059170156450233079" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } }
I20260806 00:53:04.090247 13178 leader_election.cc:302] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0477c1c737247059170156450233079; no voters:
I20260806 00:53:04.090505 13178 leader_election.cc:288] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:04.090629 13192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:04.090890 13192 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079 [term 1 LEADER]: Becoming Leader. State: Replica: d0477c1c737247059170156450233079, State: Running, Role: LEADER
I20260806 00:53:04.091213 13192 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0477c1c737247059170156450233079" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } }
I20260806 00:53:04.091687 13178 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:04.092912 13194 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0477c1c737247059170156450233079. Latest consensus state: current_term: 1 leader_uuid: "d0477c1c737247059170156450233079" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0477c1c737247059170156450233079" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } } }
I20260806 00:53:04.093051 13194 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:04.093365 13193 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0477c1c737247059170156450233079" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0477c1c737247059170156450233079" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } } }
I20260806 00:53:04.093485 13193 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:04.093432 13207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:04.096746 13207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:04.102272 13207 catalog_manager.cc:1347] Generated new cluster ID: 3a051b989f534a6fa9b1eb1355692cde
I20260806 00:53:04.102342 13207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:04.129056 13207 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:04.129426 13207 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:04.135528 13207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0477c1c737247059170156450233079: Generated new TSK 0
I20260806 00:53:04.136264 13207 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:04.143769 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 13085
2026-08-06T00:53:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1610 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:04 dist-test-slave-45dx krb5kdc[13224](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:04 dist-test-slave-45dx krb5kdc[13224](info): set up 2 sockets
Aug 06 00:53:04 dist-test-slave-45dx krb5kdc[13224](info): commencing operation
krb5kdc: starting...
W20260806 00:53:05.345280 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:05 dist-test-slave-45dx krb5kdc[13224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:53:05.484607 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39785
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:05.599421 13239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:05.599722 13239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:05.599804 13239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:05.604046 13239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:05.604138 13239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:05.604175 13239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:05.604212 13239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:05.604292 13239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:05.614477 13239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:05.615810 13239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:05.617348 13239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:05.626293 13249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:05.626282 13246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:05.626322 13245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:05.626754 13239 server_base.cc:1034] running on GCE node
I20260806 00:53:05.627364 13239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:05.628520 13239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:05.629691 13239 hybrid_clock.cc:648] HybridClock initialized: now 1785977585629672 us; error 30 us; skew 500 ppm
Aug 06 00:53:05 dist-test-slave-45dx krb5kdc[13224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977585, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:05.632606 13239 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:53:05.633638 13239 webserver.cc:460] Webserver started at http://127.0.0.1:46151/ using document root <none> and password file <none>
I20260806 00:53:05.634238 13239 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:05.634342 13239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:05.634590 13239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:05.636859 13239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "f575dab17ca74aa9b0b7b862d0493dfb"
format_stamp: "Formatted at 2026-08-06 00:53:05 on dist-test-slave-45dx"
server_key: "1e6374e7405bcfaa19b41d964e2c6c11"
server_key_iv: "e233169c410035cba7b82e5596eabc9c"
server_key_version: "encryptionkey@0"
I20260806 00:53:05.637480 13239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "f575dab17ca74aa9b0b7b862d0493dfb"
format_stamp: "Formatted at 2026-08-06 00:53:05 on dist-test-slave-45dx"
server_key: "1e6374e7405bcfaa19b41d964e2c6c11"
server_key_iv: "e233169c410035cba7b82e5596eabc9c"
server_key_version: "encryptionkey@0"
I20260806 00:53:05.641572 13239 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 00:53:05.644382 13259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:05.645905 13239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 00:53:05.646082 13239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "f575dab17ca74aa9b0b7b862d0493dfb"
format_stamp: "Formatted at 2026-08-06 00:53:05 on dist-test-slave-45dx"
server_key: "1e6374e7405bcfaa19b41d964e2c6c11"
server_key_iv: "e233169c410035cba7b82e5596eabc9c"
server_key_version: "encryptionkey@0"
I20260806 00:53:05.646224 13239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:05.661580 13239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:05.664963 13239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:05.665196 13239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:05.673352 13326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39785 every 8 connection(s)
I20260806 00:53:05.673348 13239 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39785
I20260806 00:53:05.674474 13239 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:05.677413 13330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:05.683159 13330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb: Bootstrap starting.
I20260806 00:53:05.684154 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 13239
I20260806 00:53:05.684289 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:05.684662 15950 external_mini_cluster.cc:1428] Setting key 34495ecd6a71e580339e37bc6406463b
I20260806 00:53:05.686249 13330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:05.687189 13330 log.cc:824] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:05.689576 13330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb: No bootstrap required, opened a new log
I20260806 00:53:05.693668 13330 raft_consensus.cc:348] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f575dab17ca74aa9b0b7b862d0493dfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39785 } }
I20260806 00:53:05.693895 13330 raft_consensus.cc:374] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:05.693971 13330 raft_consensus.cc:729] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f575dab17ca74aa9b0b7b862d0493dfb, State: Initialized, Role: FOLLOWER
I20260806 00:53:05.694525 13330 consensus_queue.cc:260] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f575dab17ca74aa9b0b7b862d0493dfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39785 } }
I20260806 00:53:05.694710 13330 raft_consensus.cc:388] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 06 00:53:05 dist-test-slave-45dx krb5kdc[13224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 00:53:05.694777 13330 raft_consensus.cc:482] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:05.694875 13330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:05.696121 13330 raft_consensus.cc:504] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f575dab17ca74aa9b0b7b862d0493dfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39785 } }
I20260806 00:53:05.696554 13330 leader_election.cc:302] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f575dab17ca74aa9b0b7b862d0493dfb; no voters:
I20260806 00:53:05.696957 13330 leader_election.cc:288] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:05.697077 13336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:05.697374 13336 raft_consensus.cc:686] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb [term 1 LEADER]: Becoming Leader. State: Replica: f575dab17ca74aa9b0b7b862d0493dfb, State: Running, Role: LEADER
I20260806 00:53:05.697752 13336 consensus_queue.cc:237] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f575dab17ca74aa9b0b7b862d0493dfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39785 } }
I20260806 00:53:05.698364 13330 sys_catalog.cc:564] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:05.699998 13339 sys_catalog.cc:455] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f575dab17ca74aa9b0b7b862d0493dfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f575dab17ca74aa9b0b7b862d0493dfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39785 } } }
I20260806 00:53:05.700155 13339 sys_catalog.cc:458] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:05.700004 13337 sys_catalog.cc:455] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader f575dab17ca74aa9b0b7b862d0493dfb. Latest consensus state: current_term: 1 leader_uuid: "f575dab17ca74aa9b0b7b862d0493dfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f575dab17ca74aa9b0b7b862d0493dfb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39785 } } }
I20260806 00:53:05.700434 13337 sys_catalog.cc:458] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:05.701934 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:05.685916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45495 (local address 127.0.0.1:39785)
0806 00:53:05.686446 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:05.686474 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:05.686525 (+ 51us) server_negotiation.cc:407] Connection header received
0806 00:53:05.687419 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:05.687445 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:05.687843 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:05.688220 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:05.689028 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:05.690504 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:05.691411 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:05.692175 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:05.692411 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:05.695455 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:05.695464 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:05.695482 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:05.695515 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:05.697768 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:05.698302 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:05.698308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:05.698314 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:05.698473 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:05.698829 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:05.698833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:05.698835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:05.699427 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:05.699631 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:05.700614 (+ 983us) server_negotiation.cc:299] Negotiation successful
0806 00:53:05.700943 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":162,"threads_started":1}
W20260806 00:53:05.703367 13353 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:53:05.703451 13353 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:53:05.703557 13354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:05.704556 13354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:05.705273 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:05.710672 13354 catalog_manager.cc:1347] Generated new cluster ID: 1159792fc27b440a9ff6e4b9da0a1fa1
I20260806 00:53:05.710740 13354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:05.716526 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:05.707636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56789 (local address 127.0.0.1:39785)
0806 00:53:05.707745 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:05.707749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:05.707980 (+ 231us) server_negotiation.cc:407] Connection header received
0806 00:53:05.708120 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:05.708124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:05.708173 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:05.708258 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:05.709427 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:05.710310 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:05.711286 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:05.711584 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:05.711702 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:05.713131 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:05.713134 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:05.713137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:05.713162 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:05.714778 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:05.715206 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:05.715210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:05.715212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:05.715296 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:05.715598 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:05.715602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:05.715603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:05.715783 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:05.715946 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:05.716185 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 00:53:05.716341 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 00:53:05.729250 13354 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:05.730654 13354 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:05.742357 13354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f575dab17ca74aa9b0b7b862d0493dfb: Generated new TSK 0
I20260806 00:53:05.743239 13354 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:05.754010 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 13239
2026-08-06T00:53:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1613 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:05 dist-test-slave-45dx krb5kdc[13368](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:05 dist-test-slave-45dx krb5kdc[13368](info): set up 2 sockets
Aug 06 00:53:05 dist-test-slave-45dx krb5kdc[13368](info): commencing operation
krb5kdc: starting...
W20260806 00:53:07.012709 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:07 dist-test-slave-45dx krb5kdc[13368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:53:07.139392 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41763
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:07.252496 13384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:07.252812 13384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:07.252905 13384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:07.256872 13384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:07.256968 13384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:07.257009 13384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:07.257069 13384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:07.257110 13384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:07.262346 13384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:07.263422 13384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:07.264473 13384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:07.272823 13391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:07.272833 13390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:07.272840 13393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:07.272881 13384 server_base.cc:1034] running on GCE node
I20260806 00:53:07.273703 13384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:07.274755 13384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:07.275928 13384 hybrid_clock.cc:648] HybridClock initialized: now 1785977587275900 us; error 39 us; skew 500 ppm
Aug 06 00:53:07 dist-test-slave-45dx krb5kdc[13368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977587, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:07.278825 13384 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:53:07.279615 13384 webserver.cc:460] Webserver started at http://127.0.0.1:33847/ using document root <none> and password file <none>
I20260806 00:53:07.280126 13384 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:07.280185 13384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:07.280437 13384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:07.282450 13384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "d2e89d68c7ac4968b6aac1c44184efa1"
format_stamp: "Formatted at 2026-08-06 00:53:07 on dist-test-slave-45dx"
server_key: "750df7f697d7a9197f1283275f8e5bce"
server_key_iv: "90a322ea7d13c27a2488eaeaf77d9a3c"
server_key_version: "encryptionkey@0"
I20260806 00:53:07.282968 13384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "d2e89d68c7ac4968b6aac1c44184efa1"
format_stamp: "Formatted at 2026-08-06 00:53:07 on dist-test-slave-45dx"
server_key: "750df7f697d7a9197f1283275f8e5bce"
server_key_iv: "90a322ea7d13c27a2488eaeaf77d9a3c"
server_key_version: "encryptionkey@0"
I20260806 00:53:07.286708 13384 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 00:53:07.289073 13404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:07.290283 13384 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:53:07.290442 13384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "d2e89d68c7ac4968b6aac1c44184efa1"
format_stamp: "Formatted at 2026-08-06 00:53:07 on dist-test-slave-45dx"
server_key: "750df7f697d7a9197f1283275f8e5bce"
server_key_iv: "90a322ea7d13c27a2488eaeaf77d9a3c"
server_key_version: "encryptionkey@0"
I20260806 00:53:07.290550 13384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:07.308878 13384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:07.311739 13384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:07.311959 13384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:07.319129 13384 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41763
I20260806 00:53:07.319211 13479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41763 every 8 connection(s)
I20260806 00:53:07.320128 13384 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:07.322906 13480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:07.326360 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 13384
I20260806 00:53:07.326515 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:07.326834 15950 external_mini_cluster.cc:1428] Setting key 5f27dddcbdfd83335538a90d75a471e4
I20260806 00:53:07.328956 13480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1: Bootstrap starting.
I20260806 00:53:07.331357 13480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:07.332258 13480 log.cc:824] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:07.334450 13480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1: No bootstrap required, opened a new log
I20260806 00:53:07.335922 13483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:07.328219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54895 (local address 127.0.0.1:41763)
0806 00:53:07.328693 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:07.328709 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:07.328744 (+ 35us) server_negotiation.cc:407] Connection header received
0806 00:53:07.329460 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:07.329478 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:07.329818 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:07.330091 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:07.330472 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:07.331489 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:07.332688 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:07.333291 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:07.333405 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:07.334250 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:07.334255 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:07.334266 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:07.334690 (+ 424us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:07.334745 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:07.334768 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:07.334845 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:07.335029 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 00:53:07.335249 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":138,"threads_started":1}
I20260806 00:53:07.338212 13480 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e89d68c7ac4968b6aac1c44184efa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41763 } }
I20260806 00:53:07.338435 13480 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:07.338493 13480 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e89d68c7ac4968b6aac1c44184efa1, State: Initialized, Role: FOLLOWER
I20260806 00:53:07.338853 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:07.339000 13480 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e89d68c7ac4968b6aac1c44184efa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41763 } }
I20260806 00:53:07.339170 13480 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:07.339239 13480 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:07.339335 13480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:07.340341 13480 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e89d68c7ac4968b6aac1c44184efa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41763 } }
I20260806 00:53:07.340713 13480 leader_election.cc:302] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2e89d68c7ac4968b6aac1c44184efa1; no voters:
I20260806 00:53:07.341082 13480 leader_election.cc:288] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:07.341516 13492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:07.341699 13492 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1 [term 1 LEADER]: Becoming Leader. State: Replica: d2e89d68c7ac4968b6aac1c44184efa1, State: Running, Role: LEADER
I20260806 00:53:07.342056 13492 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e89d68c7ac4968b6aac1c44184efa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41763 } }
I20260806 00:53:07.342521 13480 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:07.343299 13492 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2e89d68c7ac4968b6aac1c44184efa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e89d68c7ac4968b6aac1c44184efa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41763 } } }
I20260806 00:53:07.343489 13492 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:07.343746 13492 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2e89d68c7ac4968b6aac1c44184efa1. Latest consensus state: current_term: 1 leader_uuid: "d2e89d68c7ac4968b6aac1c44184efa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e89d68c7ac4968b6aac1c44184efa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41763 } } }
I20260806 00:53:07.343904 13492 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:07.345748 13483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:07.341091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42207 (local address 127.0.0.1:41763)
0806 00:53:07.341218 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:07.341223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:07.341419 (+ 196us) server_negotiation.cc:407] Connection header received
0806 00:53:07.341557 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:07.341560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:07.341613 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:07.341689 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:07.341989 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:07.342583 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:07.343631 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:07.343962 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:07.344043 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:07.345235 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:07.345239 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:07.345242 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:07.345256 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:07.345288 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:07.345302 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:07.345352 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:07.345555 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 00:53:07.345609 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 00:53:07.348201 13515 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:53:07.348285 13515 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:53:07.348397 13505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:07.349208 13505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:07.354581 13505 catalog_manager.cc:1347] Generated new cluster ID: d936c34645554e058b56436b4905d698
I20260806 00:53:07.354651 13505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:07.371697 13505 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:07.372689 13505 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:07.376755 13505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2e89d68c7ac4968b6aac1c44184efa1: Generated new TSK 0
I20260806 00:53:07.377434 13505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:07.390918 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 13384
2026-08-06T00:53:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1634 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:07 dist-test-slave-45dx krb5kdc[13521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:07 dist-test-slave-45dx krb5kdc[13521](info): set up 2 sockets
Aug 06 00:53:07 dist-test-slave-45dx krb5kdc[13521](info): commencing operation
krb5kdc: starting...
W20260806 00:53:08.609828 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:08 dist-test-slave-45dx krb5kdc[13521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:53:08.743712 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37227
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:08.857677 13538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:08.857932 13538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:08.857972 13538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:08.862059 13538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:08.862116 13538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:08.862136 13538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:08.862152 13538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:08.862167 13538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:08.867344 13538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:08.868351 13538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:08.869324 13538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:08.877853 13550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:08.877866 13549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:08.877871 13552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:08.878356 13538 server_base.cc:1034] running on GCE node
I20260806 00:53:08.878945 13538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:08.880002 13538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:08.881197 13538 hybrid_clock.cc:648] HybridClock initialized: now 1785977588881149 us; error 59 us; skew 500 ppm
Aug 06 00:53:08 dist-test-slave-45dx krb5kdc[13521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977588, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:08.884009 13538 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:53:08.884940 13538 webserver.cc:460] Webserver started at http://127.0.0.1:40275/ using document root <none> and password file <none>
I20260806 00:53:08.885522 13538 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:08.885617 13538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:08.885848 13538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:08.887918 13538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "69a42a9e4a6544c19e8660a621406d77"
format_stamp: "Formatted at 2026-08-06 00:53:08 on dist-test-slave-45dx"
server_key: "a8ae4857207f59eccf85882157715081"
server_key_iv: "f7867b428a42a7a34479b22bd1cff401"
server_key_version: "encryptionkey@0"
I20260806 00:53:08.888559 13538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "69a42a9e4a6544c19e8660a621406d77"
format_stamp: "Formatted at 2026-08-06 00:53:08 on dist-test-slave-45dx"
server_key: "a8ae4857207f59eccf85882157715081"
server_key_iv: "f7867b428a42a7a34479b22bd1cff401"
server_key_version: "encryptionkey@0"
I20260806 00:53:08.892338 13538 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:53:08.894814 13560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:08.896090 13538 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:53:08.896232 13538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "69a42a9e4a6544c19e8660a621406d77"
format_stamp: "Formatted at 2026-08-06 00:53:08 on dist-test-slave-45dx"
server_key: "a8ae4857207f59eccf85882157715081"
server_key_iv: "f7867b428a42a7a34479b22bd1cff401"
server_key_version: "encryptionkey@0"
I20260806 00:53:08.896390 13538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:08.901118 13538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:08.904083 13538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:08.904345 13538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:08.912361 13538 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37227
I20260806 00:53:08.912397 13634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37227 every 8 connection(s)
I20260806 00:53:08.913345 13538 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:08.916198 13635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:08.920810 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 13538
I20260806 00:53:08.920964 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:08.921264 15950 external_mini_cluster.cc:1428] Setting key 8284627d0a5573c6e5afa20b7d5b7aab
I20260806 00:53:08.922185 13635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77: Bootstrap starting.
I20260806 00:53:08.924710 13635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:08.925683 13635 log.cc:824] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:08.926693 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:08.922749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56507 (local address 127.0.0.1:37227)
0806 00:53:08.923188 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:08.923203 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:08.923238 (+ 35us) server_negotiation.cc:407] Connection header received
0806 00:53:08.923857 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:08.923883 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:08.924236 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:08.924573 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:08.925417 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:08.925422 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:08.925433 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:08.925444 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:08.925505 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:08.925524 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:08.925611 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:08.925763 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 00:53:08.925983 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":138,"threads_started":1}
I20260806 00:53:08.927894 13635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77: No bootstrap required, opened a new log
I20260806 00:53:08.929798 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:08.932578 13635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a42a9e4a6544c19e8660a621406d77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37227 } }
I20260806 00:53:08.932925 13635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:08.933005 13635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69a42a9e4a6544c19e8660a621406d77, State: Initialized, Role: FOLLOWER
I20260806 00:53:08.933548 13635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a42a9e4a6544c19e8660a621406d77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37227 } }
I20260806 00:53:08.933738 13635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:08.933743 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:08.931901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59495 (local address 127.0.0.1:37227)
0806 00:53:08.932022 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:08.932025 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:08.932254 (+ 229us) server_negotiation.cc:407] Connection header received
0806 00:53:08.932400 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:08.932403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:08.932433 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:08.932500 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:08.933326 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:08.933331 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:08.933337 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:08.933350 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:08.933387 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:08.933398 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:08.933471 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:08.933585 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 00:53:08.933619 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 00:53:08.933835 13635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:08.933936 13635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:08.935194 13635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a42a9e4a6544c19e8660a621406d77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37227 } }
I20260806 00:53:08.935557 13635 leader_election.cc:302] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69a42a9e4a6544c19e8660a621406d77; no voters:
I20260806 00:53:08.935876 13635 leader_election.cc:288] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:08.936007 13652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:08.936247 13652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77 [term 1 LEADER]: Becoming Leader. State: Replica: 69a42a9e4a6544c19e8660a621406d77, State: Running, Role: LEADER
I20260806 00:53:08.936621 13652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a42a9e4a6544c19e8660a621406d77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37227 } }
I20260806 00:53:08.937224 13635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:08.938475 13655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69a42a9e4a6544c19e8660a621406d77. Latest consensus state: current_term: 1 leader_uuid: "69a42a9e4a6544c19e8660a621406d77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a42a9e4a6544c19e8660a621406d77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37227 } } }
I20260806 00:53:08.938632 13655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:08.938476 13653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69a42a9e4a6544c19e8660a621406d77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a42a9e4a6544c19e8660a621406d77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37227 } } }
I20260806 00:53:08.938730 13653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77 [sys.catalog]: This master's current role is: LEADER
W20260806 00:53:08.942246 13672 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:53:08.942341 13672 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:53:08.942404 13665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:08.943216 13665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:08.949026 13665 catalog_manager.cc:1347] Generated new cluster ID: 068d6a52724c45d7af69f8b47c8dbea2
I20260806 00:53:08.949111 13665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:08.958900 13665 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:08.959244 13665 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:08.969341 13665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69a42a9e4a6544c19e8660a621406d77: Generated new TSK 0
I20260806 00:53:08.970079 13665 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:08.977767 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 13538
2026-08-06T00:53:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1583 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:09 dist-test-slave-45dx krb5kdc[13678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:09 dist-test-slave-45dx krb5kdc[13678](info): set up 2 sockets
Aug 06 00:53:09 dist-test-slave-45dx krb5kdc[13678](info): commencing operation
krb5kdc: starting...
W20260806 00:53:10.202580 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:10 dist-test-slave-45dx krb5kdc[13678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:53:10.339731 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39637
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:10.454957 13695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:10.455256 13695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:10.455335 13695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:10.459504 13695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:10.459596 13695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:10.459632 13695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:10.459668 13695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:10.459703 13695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:10.464726 13695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:10.465724 13695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:10.466761 13695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:10.475908 13703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:10.475860 13705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:10.475880 13702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:10.476182 13695 server_base.cc:1034] running on GCE node
I20260806 00:53:10.476872 13695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:10.477926 13695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:10.479117 13695 hybrid_clock.cc:648] HybridClock initialized: now 1785977590479101 us; error 35 us; skew 500 ppm
Aug 06 00:53:10 dist-test-slave-45dx krb5kdc[13678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977590, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:10.481913 13695 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:53:10.482757 13695 webserver.cc:460] Webserver started at http://127.0.0.1:45627/ using document root <none> and password file <none>
I20260806 00:53:10.483312 13695 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:10.483403 13695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:10.483634 13695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:10.485728 13695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "1051deb882804215bd48a44559093ee7"
format_stamp: "Formatted at 2026-08-06 00:53:10 on dist-test-slave-45dx"
server_key: "a8e07f183ba6ca575ffb65b7d2c1ccc8"
server_key_iv: "89dd62193414dc4a1a38faa3593b11cf"
server_key_version: "encryptionkey@0"
I20260806 00:53:10.486353 13695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "1051deb882804215bd48a44559093ee7"
format_stamp: "Formatted at 2026-08-06 00:53:10 on dist-test-slave-45dx"
server_key: "a8e07f183ba6ca575ffb65b7d2c1ccc8"
server_key_iv: "89dd62193414dc4a1a38faa3593b11cf"
server_key_version: "encryptionkey@0"
I20260806 00:53:10.490037 13695 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:53:10.492265 13712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:10.493434 13695 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:53:10.493587 13695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "1051deb882804215bd48a44559093ee7"
format_stamp: "Formatted at 2026-08-06 00:53:10 on dist-test-slave-45dx"
server_key: "a8e07f183ba6ca575ffb65b7d2c1ccc8"
server_key_iv: "89dd62193414dc4a1a38faa3593b11cf"
server_key_version: "encryptionkey@0"
I20260806 00:53:10.493716 13695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:10.508288 13695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:10.511350 13695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:10.511562 13695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:10.518936 13695 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39637
I20260806 00:53:10.518965 13792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39637 every 8 connection(s)
I20260806 00:53:10.519857 13695 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:10.522606 13794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:10.527019 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 13695
I20260806 00:53:10.527128 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:10.527449 15950 external_mini_cluster.cc:1428] Setting key 82ca5532118ce07d75d14f9df8ebe6e2
I20260806 00:53:10.528579 13794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7: Bootstrap starting.
I20260806 00:53:10.531297 13794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:10.532166 13794 log.cc:824] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:10.534246 13794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7: No bootstrap required, opened a new log
Aug 06 00:53:10 dist-test-slave-45dx krb5kdc[13678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 00:53:10.538398 13794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1051deb882804215bd48a44559093ee7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39637 } }
I20260806 00:53:10.538715 13794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:10.538782 13794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1051deb882804215bd48a44559093ee7, State: Initialized, Role: FOLLOWER
I20260806 00:53:10.539317 13794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1051deb882804215bd48a44559093ee7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39637 } }
I20260806 00:53:10.539496 13794 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:10.539567 13794 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:10.539659 13794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:10.540830 13794 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1051deb882804215bd48a44559093ee7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39637 } }
I20260806 00:53:10.541227 13794 leader_election.cc:302] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1051deb882804215bd48a44559093ee7; no voters:
I20260806 00:53:10.541584 13794 leader_election.cc:288] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:10.541728 13802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:10.542033 13802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7 [term 1 LEADER]: Becoming Leader. State: Replica: 1051deb882804215bd48a44559093ee7, State: Running, Role: LEADER
I20260806 00:53:10.542430 13802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1051deb882804215bd48a44559093ee7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39637 } }
I20260806 00:53:10.542901 13794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:10.543752 13802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1051deb882804215bd48a44559093ee7. Latest consensus state: current_term: 1 leader_uuid: "1051deb882804215bd48a44559093ee7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1051deb882804215bd48a44559093ee7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39637 } } }
I20260806 00:53:10.543916 13802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:10.543817 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:10.529123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:56885 (local address 127.0.0.1:39637)
0806 00:53:10.529557 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:10.529576 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:10.529619 (+ 43us) server_negotiation.cc:407] Connection header received
0806 00:53:10.530447 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:10.530470 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:10.530912 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:10.531264 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:10.532022 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:10.533391 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:10.534214 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:10.534887 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:10.535073 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:10.537941 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:10.537947 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:10.537963 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:10.537995 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:10.540041 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:10.540596 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:10.540602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:10.540607 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:10.540760 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:10.541123 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:10.541126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:10.541128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:10.541691 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:10.541931 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:10.542626 (+ 695us) server_negotiation.cc:299] Negotiation successful
0806 00:53:10.542859 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":116,"threads_started":1}
I20260806 00:53:10.544175 13805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1051deb882804215bd48a44559093ee7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1051deb882804215bd48a44559093ee7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39637 } } }
I20260806 00:53:10.544265 13805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:10.547570 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 00:53:10.548079 13822 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:53:10.548164 13822 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:53:10.548256 13812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:10.549124 13812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:10.555203 13812 catalog_manager.cc:1347] Generated new cluster ID: a3bb7139b5754a4099347f4164010e65
I20260806 00:53:10.555266 13812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:10.558837 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:10.549796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:41209 (local address 127.0.0.1:39637)
0806 00:53:10.549902 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:10.549906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:10.550109 (+ 203us) server_negotiation.cc:407] Connection header received
0806 00:53:10.550263 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:10.550266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:10.550317 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:10.550396 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:10.551100 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:10.551827 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:10.552947 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:10.553214 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:10.553271 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:10.555050 (+ 1779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:10.555053 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:10.555056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:10.555105 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:10.556929 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:10.557452 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:10.557456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:10.557457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:10.557545 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:10.557872 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:10.557875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:10.557877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:10.558061 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:10.558174 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:10.558593 (+ 419us) server_negotiation.cc:299] Negotiation successful
0806 00:53:10.558686 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:53:10.563642 13812 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:10.564757 13812 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:10.570454 13812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1051deb882804215bd48a44559093ee7: Generated new TSK 0
I20260806 00:53:10.571076 13812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:10.578501 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 13695
W20260806 00:53:10.593847 13676 connection.cc:535] client connection to 127.0.0.1:39637 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39637): Connection reset by peer (error 104)
2026-08-06T00:53:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1604 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:10 dist-test-slave-45dx krb5kdc[13839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:10 dist-test-slave-45dx krb5kdc[13839](info): set up 2 sockets
Aug 06 00:53:10 dist-test-slave-45dx krb5kdc[13839](info): commencing operation
krb5kdc: starting...
W20260806 00:53:11.800640 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:11 dist-test-slave-45dx krb5kdc[13839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:53:11.936009 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36543
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:12.051435 13854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:12.051692 13854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:12.051731 13854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:12.055717 13854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:12.055783 13854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:12.055801 13854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:12.055817 13854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:12.055860 13854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:12.061015 13854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:12.062098 13854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:12.063185 13854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:12.071622 13864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:12.071614 13861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:12.071626 13862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:12.071872 13854 server_base.cc:1034] running on GCE node
I20260806 00:53:12.072510 13854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:12.073612 13854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:12.074826 13854 hybrid_clock.cc:648] HybridClock initialized: now 1785977592074810 us; error 36 us; skew 500 ppm
Aug 06 00:53:12 dist-test-slave-45dx krb5kdc[13839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977592, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:12.077733 13854 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:53:12.078543 13854 webserver.cc:460] Webserver started at http://127.0.0.1:42447/ using document root <none> and password file <none>
I20260806 00:53:12.079043 13854 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:12.079100 13854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:12.079283 13854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:12.081351 13854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "e1386fa446dd40e3bf92f209839196c2"
format_stamp: "Formatted at 2026-08-06 00:53:12 on dist-test-slave-45dx"
server_key: "e88b5c7e61ce749d1455d0c9533183b6"
server_key_iv: "c4c3b049210875ddf773eb474dcfb42d"
server_key_version: "encryptionkey@0"
I20260806 00:53:12.081879 13854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "e1386fa446dd40e3bf92f209839196c2"
format_stamp: "Formatted at 2026-08-06 00:53:12 on dist-test-slave-45dx"
server_key: "e88b5c7e61ce749d1455d0c9533183b6"
server_key_iv: "c4c3b049210875ddf773eb474dcfb42d"
server_key_version: "encryptionkey@0"
I20260806 00:53:12.085455 13854 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 00:53:12.087675 13876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:12.088912 13854 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:53:12.089047 13854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "e1386fa446dd40e3bf92f209839196c2"
format_stamp: "Formatted at 2026-08-06 00:53:12 on dist-test-slave-45dx"
server_key: "e88b5c7e61ce749d1455d0c9533183b6"
server_key_iv: "c4c3b049210875ddf773eb474dcfb42d"
server_key_version: "encryptionkey@0"
I20260806 00:53:12.089146 13854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:12.108582 13854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:12.111960 13854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:12.112129 13854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:12.119381 13854 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36543
I20260806 00:53:12.119405 13945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36543 every 8 connection(s)
I20260806 00:53:12.120393 13854 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:12.123528 13946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:12.123605 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 13854
I20260806 00:53:12.123733 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:12.124051 15950 external_mini_cluster.cc:1428] Setting key c2a176544be45eb73e7ffae3791ba99c
I20260806 00:53:12.129909 13946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2: Bootstrap starting.
I20260806 00:53:12.132217 13946 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:12.133108 13946 log.cc:824] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:12.133251 13958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:12.125472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:48865 (local address 127.0.0.1:36543)
0806 00:53:12.125956 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:12.125976 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:12.126010 (+ 34us) server_negotiation.cc:407] Connection header received
0806 00:53:12.126822 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:12.126838 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:12.127171 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:12.127496 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:12.127894 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:12.128949 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:12.129879 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:12.130402 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:12.130576 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:12.131534 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:12.131539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:12.131550 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:12.131905 (+ 355us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:12.131979 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:12.132002 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:12.132142 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:12.132360 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 00:53:12.132501 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":181,"threads_started":1}
I20260806 00:53:12.135305 13946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2: No bootstrap required, opened a new log
I20260806 00:53:12.136201 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:12.140044 13946 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1386fa446dd40e3bf92f209839196c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36543 } }
I20260806 00:53:12.140336 13946 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:12.140398 13946 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1386fa446dd40e3bf92f209839196c2, State: Initialized, Role: FOLLOWER
I20260806 00:53:12.140873 13946 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1386fa446dd40e3bf92f209839196c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36543 } }
I20260806 00:53:12.141033 13946 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:12.141119 13946 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:12.141215 13946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:12.142468 13946 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1386fa446dd40e3bf92f209839196c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36543 } }
I20260806 00:53:12.142853 13946 leader_election.cc:302] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1386fa446dd40e3bf92f209839196c2; no voters:
I20260806 00:53:12.142923 13958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:12.138464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:54553 (local address 127.0.0.1:36543)
0806 00:53:12.138594 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:12.138601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:12.138829 (+ 228us) server_negotiation.cc:407] Connection header received
0806 00:53:12.138894 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:12.138897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:12.138943 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:12.139034 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:12.139250 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:12.139980 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:12.140860 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:12.141253 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:12.141370 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:12.142353 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:12.142356 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:12.142358 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:12.142369 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:12.142392 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:12.142402 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:12.142481 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:12.142719 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 00:53:12.142768 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:53:12.143230 13946 leader_election.cc:288] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:12.143322 13968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:12.143581 13968 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2 [term 1 LEADER]: Becoming Leader. State: Replica: e1386fa446dd40e3bf92f209839196c2, State: Running, Role: LEADER
I20260806 00:53:12.143890 13968 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1386fa446dd40e3bf92f209839196c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36543 } }
I20260806 00:53:12.144608 13946 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:12.146023 13969 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1386fa446dd40e3bf92f209839196c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1386fa446dd40e3bf92f209839196c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36543 } } }
I20260806 00:53:12.146023 13970 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1386fa446dd40e3bf92f209839196c2. Latest consensus state: current_term: 1 leader_uuid: "e1386fa446dd40e3bf92f209839196c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1386fa446dd40e3bf92f209839196c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36543 } } }
I20260806 00:53:12.146183 13969 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:12.146211 13970 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:12.146588 13981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:12.149838 13981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:12.155601 13981 catalog_manager.cc:1347] Generated new cluster ID: 21c5bfc2275344b4bbdf6b2e0582c2f7
I20260806 00:53:12.155678 13981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:12.168215 13981 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:12.169122 13981 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:12.175344 13981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1386fa446dd40e3bf92f209839196c2: Generated new TSK 0
I20260806 00:53:12.176157 13981 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:12.184136 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 13854
W20260806 00:53:12.195081 13834 connection.cc:535] client connection to 127.0.0.1:36543 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36543): Connection reset by peer (error 104)
2026-08-06T00:53:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1601 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:12 dist-test-slave-45dx krb5kdc[13998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:12 dist-test-slave-45dx krb5kdc[13998](info): set up 2 sockets
Aug 06 00:53:12 dist-test-slave-45dx krb5kdc[13998](info): commencing operation
krb5kdc: starting...
W20260806 00:53:13.400475 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:13 dist-test-slave-45dx krb5kdc[13998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:53:13.531958 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36983
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:13.643154 14012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:13.643455 14012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:13.643532 14012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:13.647519 14012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:13.647611 14012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:13.647651 14012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:13.647707 14012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:13.647742 14012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:13.653031 14012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:13.654232 14012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:13.655233 14012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:13.664183 14017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:13.664180 14020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:13.664183 14018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:13.664584 14012 server_base.cc:1034] running on GCE node
I20260806 00:53:13.665198 14012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:13.666270 14012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:13.667459 14012 hybrid_clock.cc:648] HybridClock initialized: now 1785977593667433 us; error 42 us; skew 500 ppm
Aug 06 00:53:13 dist-test-slave-45dx krb5kdc[13998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977593, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:13.670131 14012 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:53:13.670956 14012 webserver.cc:460] Webserver started at http://127.0.0.1:39109/ using document root <none> and password file <none>
I20260806 00:53:13.671504 14012 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:13.671604 14012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:13.671824 14012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:13.673836 14012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "6b34c5cfffae4decb4aa2e66446e83e1"
format_stamp: "Formatted at 2026-08-06 00:53:13 on dist-test-slave-45dx"
server_key: "64f2bdbc3225820d33fa98a2068902f0"
server_key_iv: "14b8ead28390b03ec2f46b5bed2fa118"
server_key_version: "encryptionkey@0"
I20260806 00:53:13.674418 14012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "6b34c5cfffae4decb4aa2e66446e83e1"
format_stamp: "Formatted at 2026-08-06 00:53:13 on dist-test-slave-45dx"
server_key: "64f2bdbc3225820d33fa98a2068902f0"
server_key_iv: "14b8ead28390b03ec2f46b5bed2fa118"
server_key_version: "encryptionkey@0"
I20260806 00:53:13.678040 14012 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:53:13.680344 14027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:13.681463 14012 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:53:13.681609 14012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "6b34c5cfffae4decb4aa2e66446e83e1"
format_stamp: "Formatted at 2026-08-06 00:53:13 on dist-test-slave-45dx"
server_key: "64f2bdbc3225820d33fa98a2068902f0"
server_key_iv: "14b8ead28390b03ec2f46b5bed2fa118"
server_key_version: "encryptionkey@0"
I20260806 00:53:13.681716 14012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:13.686303 14012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:13.689239 14012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:13.689429 14012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:13.696744 14012 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36983
I20260806 00:53:13.696768 14111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36983 every 8 connection(s)
I20260806 00:53:13.697659 14012 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:13.699038 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 14012
I20260806 00:53:13.699191 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:13.699510 15950 external_mini_cluster.cc:1428] Setting key 4ed89796180fa82719d0b2882ca328da
I20260806 00:53:13.700757 14112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:13.705180 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:13.700972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:34825 (local address 127.0.0.1:36983)
0806 00:53:13.701402 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:13.701421 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:13.701462 (+ 41us) server_negotiation.cc:407] Connection header received
0806 00:53:13.702319 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:13.702341 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:13.702693 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:13.703053 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:13.703799 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:13.703804 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:13.703819 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:13.703832 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:13.703895 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:13.703922 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:13.704089 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:13.704158 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 00:53:13.704329 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":110,"threads_started":1}
I20260806 00:53:13.707573 14112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1: Bootstrap starting.
I20260806 00:53:13.707861 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:13.709816 14112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:13.710703 14112 log.cc:824] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:13.711557 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:13.710006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:59971 (local address 127.0.0.1:36983)
0806 00:53:13.710090 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:13.710094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:13.710332 (+ 238us) server_negotiation.cc:407] Connection header received
0806 00:53:13.710435 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:13.710437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:13.710460 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:13.710518 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:13.711201 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:13.711205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:13.711207 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:13.711217 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:13.711240 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:13.711255 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:13.711343 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:13.711420 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 00:53:13.711458 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 00:53:13.712888 14112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1: No bootstrap required, opened a new log
I20260806 00:53:13.716063 14112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b34c5cfffae4decb4aa2e66446e83e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36983 } }
I20260806 00:53:13.716249 14112 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:13.716337 14112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b34c5cfffae4decb4aa2e66446e83e1, State: Initialized, Role: FOLLOWER
I20260806 00:53:13.716759 14112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b34c5cfffae4decb4aa2e66446e83e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36983 } }
I20260806 00:53:13.716935 14112 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:13.717017 14112 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:13.717120 14112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:13.717934 14112 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b34c5cfffae4decb4aa2e66446e83e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36983 } }
I20260806 00:53:13.718269 14112 leader_election.cc:302] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b34c5cfffae4decb4aa2e66446e83e1; no voters:
I20260806 00:53:13.718541 14112 leader_election.cc:288] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:13.718711 14125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:13.719026 14125 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1 [term 1 LEADER]: Becoming Leader. State: Replica: 6b34c5cfffae4decb4aa2e66446e83e1, State: Running, Role: LEADER
I20260806 00:53:13.719342 14125 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b34c5cfffae4decb4aa2e66446e83e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36983 } }
I20260806 00:53:13.719640 14112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:13.720908 14128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b34c5cfffae4decb4aa2e66446e83e1. Latest consensus state: current_term: 1 leader_uuid: "6b34c5cfffae4decb4aa2e66446e83e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b34c5cfffae4decb4aa2e66446e83e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36983 } } }
I20260806 00:53:13.721050 14128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:13.721340 14126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b34c5cfffae4decb4aa2e66446e83e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b34c5cfffae4decb4aa2e66446e83e1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36983 } } }
I20260806 00:53:13.721450 14126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:13.721616 14137 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:13.724864 14137 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:13.730011 14137 catalog_manager.cc:1347] Generated new cluster ID: a4db4fcf0563400da074397ac73fc01c
I20260806 00:53:13.730082 14137 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:13.762598 14137 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:13.762934 14137 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:13.768363 14137 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b34c5cfffae4decb4aa2e66446e83e1: Generated new TSK 0
I20260806 00:53:13.769083 14137 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:13.772383 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 14012
2026-08-06T00:53:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1592 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:13 dist-test-slave-45dx krb5kdc[14151](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:13 dist-test-slave-45dx krb5kdc[14151](info): set up 2 sockets
Aug 06 00:53:13 dist-test-slave-45dx krb5kdc[14151](info): commencing operation
krb5kdc: starting...
W20260806 00:53:14.983763 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:15 dist-test-slave-45dx krb5kdc[14151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:53:15.115989 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46483
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:15.225971 14170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:15.226231 14170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:15.226271 14170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:15.230305 14170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:15.230379 14170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:15.230399 14170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:15.230415 14170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:15.230430 14170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:15.235773 14170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46483
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:15.236841 14170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:15.237918 14170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:15.246694 14180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:15.246711 14184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:15.246703 14181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:15.246745 14170 server_base.cc:1034] running on GCE node
I20260806 00:53:15.247537 14170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:15.248669 14170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:15.249835 14170 hybrid_clock.cc:648] HybridClock initialized: now 1785977595249803 us; error 46 us; skew 500 ppm
Aug 06 00:53:15 dist-test-slave-45dx krb5kdc[14151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977595, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:15.260198 14170 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:53:15.261251 14170 webserver.cc:460] Webserver started at http://127.0.0.1:44493/ using document root <none> and password file <none>
I20260806 00:53:15.261862 14170 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:15.261960 14170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:15.262218 14170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:15.264441 14170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "1bfa0e95bbbe4f22b32f534e750807ec"
format_stamp: "Formatted at 2026-08-06 00:53:15 on dist-test-slave-45dx"
server_key: "61d2850ca2d466b612dd35d6fad1af50"
server_key_iv: "0f36b6809b4d33d791ac4588c15834a5"
server_key_version: "encryptionkey@0"
I20260806 00:53:15.265025 14170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "1bfa0e95bbbe4f22b32f534e750807ec"
format_stamp: "Formatted at 2026-08-06 00:53:15 on dist-test-slave-45dx"
server_key: "61d2850ca2d466b612dd35d6fad1af50"
server_key_iv: "0f36b6809b4d33d791ac4588c15834a5"
server_key_version: "encryptionkey@0"
I20260806 00:53:15.268769 14170 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:53:15.271211 14196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:15.272554 14170 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:53:15.272692 14170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "1bfa0e95bbbe4f22b32f534e750807ec"
format_stamp: "Formatted at 2026-08-06 00:53:15 on dist-test-slave-45dx"
server_key: "61d2850ca2d466b612dd35d6fad1af50"
server_key_iv: "0f36b6809b4d33d791ac4588c15834a5"
server_key_version: "encryptionkey@0"
I20260806 00:53:15.272800 14170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:15.298197 14170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:15.301117 14170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:15.301328 14170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:15.308648 14170 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46483
I20260806 00:53:15.308692 14262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46483 every 8 connection(s)
I20260806 00:53:15.309629 14170 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:15.312367 14263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:15.313165 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 14170
I20260806 00:53:15.313323 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:15.313661 15950 external_mini_cluster.cc:1428] Setting key 4bf8af2688fe4c9c38f71ffcd0fb857a
I20260806 00:53:15.319096 14263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec: Bootstrap starting.
I20260806 00:53:15.321485 14263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:15.322314 14263 log.cc:824] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:53:15 dist-test-slave-45dx krb5kdc[14151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 00:53:15.324545 14263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec: No bootstrap required, opened a new log
I20260806 00:53:15.328202 14263 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfa0e95bbbe4f22b32f534e750807ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46483 } }
I20260806 00:53:15.328542 14263 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:15.328624 14263 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bfa0e95bbbe4f22b32f534e750807ec, State: Initialized, Role: FOLLOWER
I20260806 00:53:15.328917 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:15.315115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:50937 (local address 127.0.0.1:46483)
0806 00:53:15.315589 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:15.315604 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:15.315641 (+ 37us) server_negotiation.cc:407] Connection header received
0806 00:53:15.316625 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:15.316651 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:15.317144 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:15.317504 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:15.318347 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:15.319432 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:15.320375 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:15.320860 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:15.321034 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:15.323915 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:15.323921 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:15.323935 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:15.323962 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:15.325716 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:15.326261 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:15.326267 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:15.326271 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:15.326382 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:15.326746 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:15.326754 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:15.326755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:15.327149 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:15.327409 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:15.327689 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 00:53:15.327912 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":141,"threads_started":1}
I20260806 00:53:15.329162 14263 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfa0e95bbbe4f22b32f534e750807ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46483 } }
I20260806 00:53:15.329324 14263 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:15.329394 14263 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:15.329502 14263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:15.330523 14263 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfa0e95bbbe4f22b32f534e750807ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46483 } }
I20260806 00:53:15.330938 14263 leader_election.cc:302] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1bfa0e95bbbe4f22b32f534e750807ec; no voters:
I20260806 00:53:15.331308 14263 leader_election.cc:288] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:15.331421 14272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:15.331656 14272 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec [term 1 LEADER]: Becoming Leader. State: Replica: 1bfa0e95bbbe4f22b32f534e750807ec, State: Running, Role: LEADER
I20260806 00:53:15.332056 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:15.332019 14272 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfa0e95bbbe4f22b32f534e750807ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46483 } }
I20260806 00:53:15.332667 14263 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:15.333786 14276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1bfa0e95bbbe4f22b32f534e750807ec. Latest consensus state: current_term: 1 leader_uuid: "1bfa0e95bbbe4f22b32f534e750807ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfa0e95bbbe4f22b32f534e750807ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46483 } } }
I20260806 00:53:15.333791 14275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1bfa0e95bbbe4f22b32f534e750807ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bfa0e95bbbe4f22b32f534e750807ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46483 } } }
I20260806 00:53:15.333943 14276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:15.333943 14275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:15.334326 14290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:15.337513 14290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:15.343820 14290 catalog_manager.cc:1347] Generated new cluster ID: 9e496c67a8084a01be87cbb444c9939b
I20260806 00:53:15.343885 14290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:15.346297 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:15.335145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:59511 (local address 127.0.0.1:46483)
0806 00:53:15.335265 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:15.335270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:15.336070 (+ 800us) server_negotiation.cc:407] Connection header received
0806 00:53:15.336112 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:15.336115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:15.336168 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:15.336248 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:15.337243 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:15.337846 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:15.338830 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:15.339111 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:15.339219 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:15.341569 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:15.341573 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:15.341576 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:15.341605 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:15.343734 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:15.344391 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:15.344395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:15.344406 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:15.344497 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:15.344933 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:15.344937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:15.344939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:15.345167 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:15.345325 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:15.346022 (+ 697us) server_negotiation.cc:299] Negotiation successful
0806 00:53:15.346125 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 00:53:15.353513 14290 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:15.354383 14290 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:15.359984 14290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1bfa0e95bbbe4f22b32f534e750807ec: Generated new TSK 0
I20260806 00:53:15.360755 14290 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:15.369985 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 14170
W20260806 00:53:15.386526 14148 connection.cc:535] client connection to 127.0.0.1:46483 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46483): Connection reset by peer (error 104)
2026-08-06T00:53:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1598 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:15 dist-test-slave-45dx krb5kdc[14303](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:15 dist-test-slave-45dx krb5kdc[14303](info): set up 2 sockets
Aug 06 00:53:15 dist-test-slave-45dx krb5kdc[14303](info): commencing operation
krb5kdc: starting...
W20260806 00:53:16.579809 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:16 dist-test-slave-45dx krb5kdc[14303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:53:16.707482 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35731
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:16.817857 14323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:16.818216 14323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:16.818301 14323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:16.822592 14323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:16.822659 14323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:16.822677 14323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:16.822695 14323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:16.822710 14323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:16.827848 14323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35731
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:16.828888 14323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:16.829850 14323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:16.838456 14334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:16.838495 14328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:16.838495 14329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:16.838640 14323 server_base.cc:1034] running on GCE node
I20260806 00:53:16.839398 14323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:16.840585 14323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:16.841784 14323 hybrid_clock.cc:648] HybridClock initialized: now 1785977596841769 us; error 35 us; skew 500 ppm
Aug 06 00:53:16 dist-test-slave-45dx krb5kdc[14303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977596, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:16.844589 14323 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:53:16.845564 14323 webserver.cc:460] Webserver started at http://127.0.0.1:36499/ using document root <none> and password file <none>
I20260806 00:53:16.846140 14323 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:16.846237 14323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:16.846477 14323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:16.848623 14323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "aa1f6219b73e46f781fd36ba630ffc06"
format_stamp: "Formatted at 2026-08-06 00:53:16 on dist-test-slave-45dx"
server_key: "97d2db3df7a560a2041f5a86c4f63857"
server_key_iv: "945c164b7733f924ccd3fe5693f594c0"
server_key_version: "encryptionkey@0"
I20260806 00:53:16.849186 14323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "aa1f6219b73e46f781fd36ba630ffc06"
format_stamp: "Formatted at 2026-08-06 00:53:16 on dist-test-slave-45dx"
server_key: "97d2db3df7a560a2041f5a86c4f63857"
server_key_iv: "945c164b7733f924ccd3fe5693f594c0"
server_key_version: "encryptionkey@0"
I20260806 00:53:16.852742 14323 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:53:16.854962 14343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:16.856081 14323 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:53:16.856218 14323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "aa1f6219b73e46f781fd36ba630ffc06"
format_stamp: "Formatted at 2026-08-06 00:53:16 on dist-test-slave-45dx"
server_key: "97d2db3df7a560a2041f5a86c4f63857"
server_key_iv: "945c164b7733f924ccd3fe5693f594c0"
server_key_version: "encryptionkey@0"
I20260806 00:53:16.856344 14323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:16.877105 14323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:16.879840 14323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:16.880055 14323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:16.887068 14323 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35731
I20260806 00:53:16.887393 14416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35731 every 8 connection(s)
I20260806 00:53:16.888070 14323 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:16.890954 14418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:16.894798 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 14323
I20260806 00:53:16.894979 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:16.895291 15950 external_mini_cluster.cc:1428] Setting key bdf8f117dd8f4a882e3570aceedc127d
I20260806 00:53:16.897154 14418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06: Bootstrap starting.
I20260806 00:53:16.899581 14418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:16.900534 14418 log.cc:824] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:16.902889 14418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06: No bootstrap required, opened a new log
I20260806 00:53:16.904982 14425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:16.896757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:35887 (local address 127.0.0.1:35731)
0806 00:53:16.897199 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:16.897214 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:16.897251 (+ 37us) server_negotiation.cc:407] Connection header received
0806 00:53:16.897926 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:16.897943 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:16.898280 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:16.898572 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:16.898929 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:16.899968 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:16.901200 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:16.901765 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:16.901888 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:16.902849 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:16.902855 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:16.902868 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:16.903337 (+ 469us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:16.903391 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:16.903419 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:16.903537 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:16.903750 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 00:53:16.903933 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":131,"threads_started":1}
I20260806 00:53:16.906523 14418 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa1f6219b73e46f781fd36ba630ffc06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35731 } }
I20260806 00:53:16.906770 14418 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:16.906833 14418 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa1f6219b73e46f781fd36ba630ffc06, State: Initialized, Role: FOLLOWER
I20260806 00:53:16.907248 14418 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa1f6219b73e46f781fd36ba630ffc06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35731 } }
I20260806 00:53:16.907401 14418 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:16.907485 14418 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:16.907612 14418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:16.908288 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:16.908795 14418 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa1f6219b73e46f781fd36ba630ffc06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35731 } }
I20260806 00:53:16.909129 14418 leader_election.cc:302] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa1f6219b73e46f781fd36ba630ffc06; no voters:
I20260806 00:53:16.909436 14418 leader_election.cc:288] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:16.909596 14428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:16.909881 14428 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06 [term 1 LEADER]: Becoming Leader. State: Replica: aa1f6219b73e46f781fd36ba630ffc06, State: Running, Role: LEADER
I20260806 00:53:16.910228 14428 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa1f6219b73e46f781fd36ba630ffc06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35731 } }
I20260806 00:53:16.910708 14418 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:16.911262 14428 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa1f6219b73e46f781fd36ba630ffc06. Latest consensus state: current_term: 1 leader_uuid: "aa1f6219b73e46f781fd36ba630ffc06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa1f6219b73e46f781fd36ba630ffc06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35731 } } }
I20260806 00:53:16.911389 14428 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:16.911715 14432 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa1f6219b73e46f781fd36ba630ffc06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa1f6219b73e46f781fd36ba630ffc06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35731 } } }
I20260806 00:53:16.911813 14432 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:16.911871 14440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:16.915069 14440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:16.918195 14425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:16.913383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:48815 (local address 127.0.0.1:35731)
0806 00:53:16.913508 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:16.913512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:16.914150 (+ 638us) server_negotiation.cc:407] Connection header received
0806 00:53:16.914295 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:16.914298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:16.914347 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:16.914438 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:16.914712 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:16.915465 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:16.916358 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:16.916651 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:16.916772 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:16.917681 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:16.917685 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:16.917688 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:16.917701 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:16.917729 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:16.917743 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:16.917840 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:16.918023 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 00:53:16.918060 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 00:53:16.921142 14440 catalog_manager.cc:1347] Generated new cluster ID: 5ceb3e3e6b0b4862aaa4d1db54c0abeb
I20260806 00:53:16.921216 14440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:16.927613 14440 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:16.928565 14440 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:16.939110 14440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa1f6219b73e46f781fd36ba630ffc06: Generated new TSK 0
I20260806 00:53:16.939810 14440 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:16.948844 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 14323
W20260806 00:53:16.967010 14301 connection.cc:535] client connection to 127.0.0.1:35731 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35731): Connection reset by peer (error 104)
2026-08-06T00:53:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1580 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:17 dist-test-slave-45dx krb5kdc[14464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:17 dist-test-slave-45dx krb5kdc[14464](info): set up 2 sockets
Aug 06 00:53:17 dist-test-slave-45dx krb5kdc[14464](info): commencing operation
krb5kdc: starting...
W20260806 00:53:18.182359 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:18 dist-test-slave-45dx krb5kdc[14464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 00:53:18.311842 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39841
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:18.422312 14485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:18.422569 14485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:18.422610 14485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:18.426556 14485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:18.426615 14485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:18.426635 14485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:18.426649 14485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:18.426664 14485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:18.431905 14485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39841
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:18.432960 14485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:18.433954 14485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:18.442399 14497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:18.442420 14495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:18.442399 14493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:18.442915 14485 server_base.cc:1034] running on GCE node
I20260806 00:53:18.443459 14485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:18.444515 14485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:18.445703 14485 hybrid_clock.cc:648] HybridClock initialized: now 1785977598445687 us; error 34 us; skew 500 ppm
Aug 06 00:53:18 dist-test-slave-45dx krb5kdc[14464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977598, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:18.448616 14485 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 00:53:18.449460 14485 webserver.cc:460] Webserver started at http://127.0.0.1:46507/ using document root <none> and password file <none>
I20260806 00:53:18.449999 14485 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:18.450089 14485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:18.450294 14485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:18.452248 14485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "7204e9608cc645aa979038a149ba395c"
format_stamp: "Formatted at 2026-08-06 00:53:18 on dist-test-slave-45dx"
server_key: "1396584a29e390c56922369b5781a9c4"
server_key_iv: "e0e20fcd50ecb94da533aa34be69d570"
server_key_version: "encryptionkey@0"
I20260806 00:53:18.452860 14485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "7204e9608cc645aa979038a149ba395c"
format_stamp: "Formatted at 2026-08-06 00:53:18 on dist-test-slave-45dx"
server_key: "1396584a29e390c56922369b5781a9c4"
server_key_iv: "e0e20fcd50ecb94da533aa34be69d570"
server_key_version: "encryptionkey@0"
I20260806 00:53:18.456499 14485 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 00:53:18.458886 14507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:18.460008 14485 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:53:18.460148 14485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "7204e9608cc645aa979038a149ba395c"
format_stamp: "Formatted at 2026-08-06 00:53:18 on dist-test-slave-45dx"
server_key: "1396584a29e390c56922369b5781a9c4"
server_key_iv: "e0e20fcd50ecb94da533aa34be69d570"
server_key_version: "encryptionkey@0"
I20260806 00:53:18.460258 14485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:18.464762 14485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:18.467453 14485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:18.467641 14485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:18.475198 14485 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39841
I20260806 00:53:18.475226 14575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39841 every 8 connection(s)
I20260806 00:53:18.476022 14485 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:18.478224 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 14485
I20260806 00:53:18.478380 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:18.478684 15950 external_mini_cluster.cc:1428] Setting key 39bc726003c9baef43081cb17dab83ee
I20260806 00:53:18.478838 14576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:18.483896 14579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:18.479999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:47239 (local address 127.0.0.1:39841)
0806 00:53:18.480488 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:18.480502 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:18.480536 (+ 34us) server_negotiation.cc:407] Connection header received
0806 00:53:18.481332 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:18.481349 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:18.481609 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:18.481960 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:18.482781 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:18.482785 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:18.482796 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:18.482807 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:18.482860 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:18.482878 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:18.483042 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:18.483109 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 00:53:18.483265 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":148,"threads_started":1}
I20260806 00:53:18.485488 14576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c: Bootstrap starting.
I20260806 00:53:18.486734 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:18.488121 14576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:18.489023 14576 log.cc:824] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:18.490707 14579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:18.489079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.37:34963 (local address 127.0.0.1:39841)
0806 00:53:18.489198 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:18.489202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:18.489416 (+ 214us) server_negotiation.cc:407] Connection header received
0806 00:53:18.489548 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:18.489552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:18.489589 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:18.489671 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:18.490331 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:18.490334 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:18.490337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:18.490348 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:18.490374 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:18.490389 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:18.490483 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:18.490556 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 00:53:18.490594 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 00:53:18.491132 14576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c: No bootstrap required, opened a new log
I20260806 00:53:18.494808 14576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7204e9608cc645aa979038a149ba395c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39841 } }
I20260806 00:53:18.495016 14576 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:18.495090 14576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7204e9608cc645aa979038a149ba395c, State: Initialized, Role: FOLLOWER
I20260806 00:53:18.495473 14576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7204e9608cc645aa979038a149ba395c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39841 } }
I20260806 00:53:18.495604 14576 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:18.495669 14576 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:18.495760 14576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:18.496644 14576 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7204e9608cc645aa979038a149ba395c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39841 } }
I20260806 00:53:18.496975 14576 leader_election.cc:302] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7204e9608cc645aa979038a149ba395c; no voters:
I20260806 00:53:18.497280 14576 leader_election.cc:288] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:18.497424 14590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:18.497651 14590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c [term 1 LEADER]: Becoming Leader. State: Replica: 7204e9608cc645aa979038a149ba395c, State: Running, Role: LEADER
I20260806 00:53:18.497942 14590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7204e9608cc645aa979038a149ba395c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39841 } }
I20260806 00:53:18.498447 14576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:18.499583 14592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7204e9608cc645aa979038a149ba395c. Latest consensus state: current_term: 1 leader_uuid: "7204e9608cc645aa979038a149ba395c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7204e9608cc645aa979038a149ba395c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39841 } } }
I20260806 00:53:18.499717 14592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:18.500031 14591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7204e9608cc645aa979038a149ba395c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7204e9608cc645aa979038a149ba395c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39841 } } }
I20260806 00:53:18.500123 14591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:18.500211 14605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:18.503334 14605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:18.508396 14605 catalog_manager.cc:1347] Generated new cluster ID: 9467892a012f487aa81c300e7c733f65
I20260806 00:53:18.508468 14605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:18.525658 14605 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:18.525992 14605 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:18.538657 14605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7204e9608cc645aa979038a149ba395c: Generated new TSK 0
I20260806 00:53:18.539413 14605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:18.548559 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 14485
2026-08-06T00:53:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1591 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:18 dist-test-slave-45dx krb5kdc[14622](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:18 dist-test-slave-45dx krb5kdc[14622](info): set up 2 sockets
Aug 06 00:53:18 dist-test-slave-45dx krb5kdc[14622](info): commencing operation
krb5kdc: starting...
W20260806 00:53:19.757704 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:19 dist-test-slave-45dx krb5kdc[14622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:53:19.885676 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:35893
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:44441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:35893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:19.999485 14639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:19.999789 14639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:19.999871 14639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:20.003940 14639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:20.004050 14639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:20.004087 14639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:20.004127 14639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:20.004163 14639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:20.009596 14639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:44441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:35893
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:35893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:20.010710 14639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:20.011780 14639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:53:20.020692 14639 server_base.cc:1034] running on GCE node
W20260806 00:53:20.020642 14647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:20.020624 14648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:20.020604 14650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:20.021407 14639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:20.022470 14639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:20.023638 14639 hybrid_clock.cc:648] HybridClock initialized: now 1785977600023610 us; error 41 us; skew 500 ppm
Aug 06 00:53:20 dist-test-slave-45dx krb5kdc[14622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977600, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:20.026636 14639 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:53:20.027529 14639 webserver.cc:460] Webserver started at http://127.15.147.190:42263/ using document root <none> and password file <none>
I20260806 00:53:20.028084 14639 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:20.028179 14639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:20.028467 14639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:20.030560 14639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "849db649b1194b82860d1cb745153f44"
format_stamp: "Formatted at 2026-08-06 00:53:20 on dist-test-slave-45dx"
server_key: "336cc7b58c46d0ed16ef0c600a03e614"
server_key_iv: "bc4fd788efb053f951a6e252d665d408"
server_key_version: "encryptionkey@0"
I20260806 00:53:20.031220 14639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "849db649b1194b82860d1cb745153f44"
format_stamp: "Formatted at 2026-08-06 00:53:20 on dist-test-slave-45dx"
server_key: "336cc7b58c46d0ed16ef0c600a03e614"
server_key_iv: "bc4fd788efb053f951a6e252d665d408"
server_key_version: "encryptionkey@0"
I20260806 00:53:20.035295 14639 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 00:53:20.037941 14658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:20.039208 14639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:53:20.039363 14639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "849db649b1194b82860d1cb745153f44"
format_stamp: "Formatted at 2026-08-06 00:53:20 on dist-test-slave-45dx"
server_key: "336cc7b58c46d0ed16ef0c600a03e614"
server_key_iv: "bc4fd788efb053f951a6e252d665d408"
server_key_version: "encryptionkey@0"
I20260806 00:53:20.039482 14639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:20.050041 14639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:20.053063 14639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:20.053275 14639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:20.061062 14639 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:35893
I20260806 00:53:20.061089 14731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:35893 every 8 connection(s)
I20260806 00:53:20.062064 14639 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:20.064920 14732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:20.070529 14732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44: Bootstrap starting.
I20260806 00:53:20.072429 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 14639
I20260806 00:53:20.072585 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:20.072916 15950 external_mini_cluster.cc:1428] Setting key 1946ed9fa66cfac73cc5264a2029cc3e
I20260806 00:53:20.073343 14732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:20.074198 14732 log.cc:824] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:20.076145 14732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44: No bootstrap required, opened a new log
I20260806 00:53:20.079593 14732 raft_consensus.cc:348] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "849db649b1194b82860d1cb745153f44" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35893 } }
I20260806 00:53:20.079845 14732 raft_consensus.cc:374] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:20.079974 14732 raft_consensus.cc:729] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 849db649b1194b82860d1cb745153f44, State: Initialized, Role: FOLLOWER
I20260806 00:53:20.080577 14732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "849db649b1194b82860d1cb745153f44" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35893 } }
I20260806 00:53:20.080767 14732 raft_consensus.cc:388] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:20.080845 14732 raft_consensus.cc:482] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:20.080956 14732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 00:53:20 dist-test-slave-45dx krb5kdc[14622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:53:20.082211 14732 raft_consensus.cc:504] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "849db649b1194b82860d1cb745153f44" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35893 } }
I20260806 00:53:20.082556 14732 leader_election.cc:302] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 849db649b1194b82860d1cb745153f44; no voters:
I20260806 00:53:20.082918 14732 leader_election.cc:288] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:20.083033 14740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:20.083302 14740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44 [term 1 LEADER]: Becoming Leader. State: Replica: 849db649b1194b82860d1cb745153f44, State: Running, Role: LEADER
I20260806 00:53:20.083630 14740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "849db649b1194b82860d1cb745153f44" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35893 } }
I20260806 00:53:20.084403 14732 sys_catalog.cc:564] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:20.085457 14741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "849db649b1194b82860d1cb745153f44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "849db649b1194b82860d1cb745153f44" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35893 } } }
I20260806 00:53:20.085527 14743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 849db649b1194b82860d1cb745153f44. Latest consensus state: current_term: 1 leader_uuid: "849db649b1194b82860d1cb745153f44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "849db649b1194b82860d1cb745153f44" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 35893 } } }
I20260806 00:53:20.085611 14741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:20.085632 14743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:20.086000 14752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:20.089228 14752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:20.089301 14736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:20.074155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45173 (local address 127.15.147.190:35893)
0806 00:53:20.074662 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:20.074683 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:20.074726 (+ 43us) server_negotiation.cc:407] Connection header received
0806 00:53:20.075469 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:20.075487 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:20.075806 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:20.076143 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:20.077013 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:20.077985 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:20.078892 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:20.079389 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:20.079565 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:20.082868 (+ 3303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:20.082876 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:20.082894 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:20.082927 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:20.085210 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:20.086392 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:20.086398 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:20.086404 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:20.086547 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:20.086974 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:20.086977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:20.086979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:20.087499 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:20.087738 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:20.088009 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 00:53:20.088270 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":142,"threads_started":1}
I20260806 00:53:20.092617 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:20.096779 14752 catalog_manager.cc:1347] Generated new cluster ID: 9f78a2b682b543c8bf79e5a0d6fc11b8
I20260806 00:53:20.096863 14752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:20.103821 14736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:20.095128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35577 (local address 127.15.147.190:35893)
0806 00:53:20.095236 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:20.095240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:20.095545 (+ 305us) server_negotiation.cc:407] Connection header received
0806 00:53:20.095671 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:20.095674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:20.095724 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:20.095815 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:20.096770 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:20.097388 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:20.098250 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:20.098524 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:20.098653 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:20.100196 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:20.100200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:20.100203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:20.100233 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:20.101895 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:20.102416 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:20.102420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:20.102421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:20.102517 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:20.102869 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:20.102873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:20.102875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:20.103071 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:20.103252 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:20.103535 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 00:53:20.103643 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 00:53:20.109776 14752 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:20.110795 14752 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:20.118453 14752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 849db649b1194b82860d1cb745153f44: Generated new TSK 0
I20260806 00:53:20.119638 14752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:20.124943 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 14639
W20260806 00:53:20.139027 14618 connection.cc:535] client connection to 127.15.147.190:35893 recv error: Network error: failed to read from TLS socket (remote: 127.15.147.190:35893): Connection reset by peer (error 104)
2026-08-06T00:53:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1580 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:20 dist-test-slave-45dx krb5kdc[14774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:20 dist-test-slave-45dx krb5kdc[14774](info): set up 2 sockets
Aug 06 00:53:20 dist-test-slave-45dx krb5kdc[14774](info): commencing operation
krb5kdc: starting...
W20260806 00:53:21.353451 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:21 dist-test-slave-45dx krb5kdc[14774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:53:21.489424 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:38907
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:33361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:38907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:21.602720 14795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:21.603046 14795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:21.603132 14795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:21.607106 14795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:21.607203 14795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:21.607241 14795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:21.607285 14795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:21.607323 14795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:21.612490 14795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:33361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:38907
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:38907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:21.613654 14795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:21.614742 14795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:21.623718 14806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:21.623713 14803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:21.623873 14795 server_base.cc:1034] running on GCE node
W20260806 00:53:21.623824 14804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:21.624691 14795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:21.625697 14795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:21.626868 14795 hybrid_clock.cc:648] HybridClock initialized: now 1785977601626850 us; error 30 us; skew 500 ppm
Aug 06 00:53:21 dist-test-slave-45dx krb5kdc[14774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977601, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:21.635392 14795 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:53:21.636453 14795 webserver.cc:460] Webserver started at http://127.15.147.190:39419/ using document root <none> and password file <none>
I20260806 00:53:21.636996 14795 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:21.637055 14795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:21.637312 14795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:21.639413 14795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "f1bfdae5c8404fd3abab10b2e533db3c"
format_stamp: "Formatted at 2026-08-06 00:53:21 on dist-test-slave-45dx"
server_key: "dd3b093447a207d725326c8ebefeae4d"
server_key_iv: "b3feb8a45f437446557cb24ee297b65e"
server_key_version: "encryptionkey@0"
I20260806 00:53:21.639962 14795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "f1bfdae5c8404fd3abab10b2e533db3c"
format_stamp: "Formatted at 2026-08-06 00:53:21 on dist-test-slave-45dx"
server_key: "dd3b093447a207d725326c8ebefeae4d"
server_key_iv: "b3feb8a45f437446557cb24ee297b65e"
server_key_version: "encryptionkey@0"
I20260806 00:53:21.643621 14795 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 00:53:21.645902 14816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:21.647074 14795 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:53:21.647208 14795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "f1bfdae5c8404fd3abab10b2e533db3c"
format_stamp: "Formatted at 2026-08-06 00:53:21 on dist-test-slave-45dx"
server_key: "dd3b093447a207d725326c8ebefeae4d"
server_key_iv: "b3feb8a45f437446557cb24ee297b65e"
server_key_version: "encryptionkey@0"
I20260806 00:53:21.647324 14795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:21.660388 14795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:21.662878 14795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:21.663100 14795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:21.670140 14795 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:38907
I20260806 00:53:21.670197 14884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:38907 every 8 connection(s)
I20260806 00:53:21.671022 14795 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:21.673833 14885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:21.676841 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 14795
I20260806 00:53:21.677006 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:21.677304 15950 external_mini_cluster.cc:1428] Setting key f711231e6d882dfd0f1846a494d48467
I20260806 00:53:21.679733 14885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c: Bootstrap starting.
I20260806 00:53:21.681938 14885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:21.682652 14885 log.cc:824] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:21.684609 14885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c: No bootstrap required, opened a new log
I20260806 00:53:21.685932 14892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:21.678706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37463 (local address 127.15.147.190:38907)
0806 00:53:21.679165 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:21.679180 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:21.679214 (+ 34us) server_negotiation.cc:407] Connection header received
0806 00:53:21.679832 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:21.679857 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:21.680186 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:21.680552 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:21.680958 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:21.681911 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:21.682762 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:21.683215 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:21.683394 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:21.684334 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:21.684342 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:21.684357 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:21.684745 (+ 388us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:21.684814 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:21.684837 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:21.684971 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:21.685153 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 00:53:21.685286 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":145,"threads_started":1}
I20260806 00:53:21.688687 14885 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1bfdae5c8404fd3abab10b2e533db3c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 38907 } }
I20260806 00:53:21.688949 14885 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:21.689085 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:21.689116 14885 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bfdae5c8404fd3abab10b2e533db3c, State: Initialized, Role: FOLLOWER
I20260806 00:53:21.689569 14885 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1bfdae5c8404fd3abab10b2e533db3c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 38907 } }
I20260806 00:53:21.689719 14885 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:21.689788 14885 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:21.689885 14885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:21.690860 14885 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1bfdae5c8404fd3abab10b2e533db3c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 38907 } }
I20260806 00:53:21.691249 14885 leader_election.cc:302] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1bfdae5c8404fd3abab10b2e533db3c; no voters:
I20260806 00:53:21.691721 14885 leader_election.cc:288] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:21.691864 14903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:21.692093 14903 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c [term 1 LEADER]: Becoming Leader. State: Replica: f1bfdae5c8404fd3abab10b2e533db3c, State: Running, Role: LEADER
I20260806 00:53:21.692519 14903 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1bfdae5c8404fd3abab10b2e533db3c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 38907 } }
I20260806 00:53:21.693130 14885 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:21.695091 14905 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1bfdae5c8404fd3abab10b2e533db3c. Latest consensus state: current_term: 1 leader_uuid: "f1bfdae5c8404fd3abab10b2e533db3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1bfdae5c8404fd3abab10b2e533db3c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 38907 } } }
I20260806 00:53:21.695221 14905 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:21.695626 14906 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1bfdae5c8404fd3abab10b2e533db3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1bfdae5c8404fd3abab10b2e533db3c" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 38907 } } }
I20260806 00:53:21.695914 14928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:21.696106 14906 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:21.696283 14892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:21.691260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38129 (local address 127.15.147.190:38907)
0806 00:53:21.691398 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:21.691402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:21.691529 (+ 127us) server_negotiation.cc:407] Connection header received
0806 00:53:21.691607 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:21.691613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:21.691681 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:21.691767 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:21.692079 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:21.692851 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:21.694001 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:21.694321 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:21.694432 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:21.695408 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:21.695412 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:21.695415 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:21.695427 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:21.695452 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:21.695466 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:21.695557 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:21.696062 (+ 505us) server_negotiation.cc:299] Negotiation successful
0806 00:53:21.696101 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 00:53:21.699265 14928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:21.704830 14928 catalog_manager.cc:1347] Generated new cluster ID: 86b898144f3a4513a04d7706fe298ef9
I20260806 00:53:21.704958 14928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:21.720569 14928 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:21.721477 14928 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:21.735199 14928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1bfdae5c8404fd3abab10b2e533db3c: Generated new TSK 0
I20260806 00:53:21.735846 14928 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:21.747709 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 14795
W20260806 00:53:21.759361 14769 connection.cc:535] client connection to 127.15.147.190:38907 recv error: Network error: failed to read from TLS socket (remote: 127.15.147.190:38907): Connection reset by peer (error 104)
2026-08-06T00:53:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1619 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:21 dist-test-slave-45dx krb5kdc[14935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:21 dist-test-slave-45dx krb5kdc[14935](info): set up 2 sockets
Aug 06 00:53:21 dist-test-slave-45dx krb5kdc[14935](info): commencing operation
krb5kdc: starting...
W20260806 00:53:22.972175 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:23 dist-test-slave-45dx krb5kdc[14935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:53:23.103003 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:41339
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:35781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:41339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:23.212179 14953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:23.212551 14953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:23.212625 14953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:23.216421 14953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:23.216514 14953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:23.216557 14953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:23.216598 14953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:23.216631 14953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:23.221690 14953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:35781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:41339
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:41339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:23.222685 14953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:23.223706 14953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:23.232125 14964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:23.232111 14961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:23.232095 14960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:23.232611 14953 server_base.cc:1034] running on GCE node
I20260806 00:53:23.233165 14953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:23.234191 14953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:23.235419 14953 hybrid_clock.cc:648] HybridClock initialized: now 1785977603235390 us; error 39 us; skew 500 ppm
Aug 06 00:53:23 dist-test-slave-45dx krb5kdc[14935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977603, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:23.238245 14953 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:53:23.239158 14953 webserver.cc:460] Webserver started at http://127.15.147.190:40163/ using document root <none> and password file <none>
I20260806 00:53:23.239686 14953 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:23.239748 14953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:23.240001 14953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:23.242007 14953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "8ad1ef73720143058cf913a964f96bd5"
format_stamp: "Formatted at 2026-08-06 00:53:23 on dist-test-slave-45dx"
server_key: "b724024b02ce342871754dbe1d19b157"
server_key_iv: "6659194a9a3036c431b342810b5b4d03"
server_key_version: "encryptionkey@0"
I20260806 00:53:23.242555 14953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "8ad1ef73720143058cf913a964f96bd5"
format_stamp: "Formatted at 2026-08-06 00:53:23 on dist-test-slave-45dx"
server_key: "b724024b02ce342871754dbe1d19b157"
server_key_iv: "6659194a9a3036c431b342810b5b4d03"
server_key_version: "encryptionkey@0"
I20260806 00:53:23.246163 14953 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 00:53:23.248520 14977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:23.249701 14953 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:53:23.249871 14953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "8ad1ef73720143058cf913a964f96bd5"
format_stamp: "Formatted at 2026-08-06 00:53:23 on dist-test-slave-45dx"
server_key: "b724024b02ce342871754dbe1d19b157"
server_key_iv: "6659194a9a3036c431b342810b5b4d03"
server_key_version: "encryptionkey@0"
I20260806 00:53:23.249964 14953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:23.254848 14953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:23.258082 14953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:23.258277 14953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:23.265591 14953 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:41339
I20260806 00:53:23.265614 15044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:41339 every 8 connection(s)
I20260806 00:53:23.266491 14953 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:23.268610 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 14953
I20260806 00:53:23.268766 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:23.269094 15950 external_mini_cluster.cc:1428] Setting key 9d0e286128e41e025b5f679437339b7d
I20260806 00:53:23.269541 15045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:23.274371 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:23.270576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50553 (local address 127.15.147.190:41339)
0806 00:53:23.270985 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:23.271000 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:23.271035 (+ 35us) server_negotiation.cc:407] Connection header received
0806 00:53:23.271657 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:23.271671 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:23.271925 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:23.272194 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:23.273172 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:23.273177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:23.273189 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:23.273199 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:23.273270 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:23.273289 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:23.273397 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:23.273505 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 00:53:23.273669 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":141,"threads_started":1}
I20260806 00:53:23.277042 15045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5: Bootstrap starting.
I20260806 00:53:23.277233 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:23.279556 15045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:23.280349 15045 log.cc:824] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:23.281031 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:23.279227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40877 (local address 127.15.147.190:41339)
0806 00:53:23.279332 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:23.279335 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:23.279575 (+ 240us) server_negotiation.cc:407] Connection header received
0806 00:53:23.279705 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:23.279709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:23.279746 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:23.279823 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:23.280574 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:23.280579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:23.280586 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:23.280596 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:23.280620 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:23.280630 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:23.280695 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:23.280840 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 00:53:23.280878 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 00:53:23.282675 15045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5: No bootstrap required, opened a new log
I20260806 00:53:23.286375 15045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad1ef73720143058cf913a964f96bd5" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41339 } }
I20260806 00:53:23.286602 15045 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:23.286664 15045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ad1ef73720143058cf913a964f96bd5, State: Initialized, Role: FOLLOWER
I20260806 00:53:23.287036 15045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad1ef73720143058cf913a964f96bd5" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41339 } }
I20260806 00:53:23.287181 15045 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:23.287264 15045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:23.287358 15045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:23.288225 15045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad1ef73720143058cf913a964f96bd5" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41339 } }
I20260806 00:53:23.288582 15045 leader_election.cc:302] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ad1ef73720143058cf913a964f96bd5; no voters:
I20260806 00:53:23.288902 15045 leader_election.cc:288] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:23.289013 15059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:23.289249 15059 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5 [term 1 LEADER]: Becoming Leader. State: Replica: 8ad1ef73720143058cf913a964f96bd5, State: Running, Role: LEADER
I20260806 00:53:23.289613 15059 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad1ef73720143058cf913a964f96bd5" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41339 } }
I20260806 00:53:23.290211 15045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:23.291436 15060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ad1ef73720143058cf913a964f96bd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad1ef73720143058cf913a964f96bd5" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41339 } } }
I20260806 00:53:23.291469 15061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ad1ef73720143058cf913a964f96bd5. Latest consensus state: current_term: 1 leader_uuid: "8ad1ef73720143058cf913a964f96bd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad1ef73720143058cf913a964f96bd5" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41339 } } }
I20260806 00:53:23.291567 15060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:23.291574 15061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:23.291966 15071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:23.294518 15071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:23.299657 15071 catalog_manager.cc:1347] Generated new cluster ID: 7de2ea8c06d048f9b1a14db92e2bb50f
I20260806 00:53:23.299726 15071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:23.315027 15071 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:23.315339 15071 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:23.321251 15071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ad1ef73720143058cf913a964f96bd5: Generated new TSK 0
I20260806 00:53:23.321899 15071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:23.327116 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 14953
2026-08-06T00:53:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1579 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:23 dist-test-slave-45dx krb5kdc[15085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:23 dist-test-slave-45dx krb5kdc[15085](info): set up 2 sockets
Aug 06 00:53:23 dist-test-slave-45dx krb5kdc[15085](info): commencing operation
krb5kdc: starting...
W20260806 00:53:24.539074 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:24 dist-test-slave-45dx krb5kdc[15085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:53:24.667434 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:41735
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:35559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:41735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:24.777462 15111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:24.777787 15111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:24.777878 15111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:24.781919 15111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:24.782011 15111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:24.782056 15111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:24.782091 15111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:24.782122 15111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:24.787240 15111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:35559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:41735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:41735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:24.788272 15111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:24.789291 15111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:24.797533 15120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:24.797570 15117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:24.797703 15111 server_base.cc:1034] running on GCE node
W20260806 00:53:24.797549 15118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:24.798494 15111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:24.799468 15111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:24.800655 15111 hybrid_clock.cc:648] HybridClock initialized: now 1785977604800626 us; error 39 us; skew 500 ppm
Aug 06 00:53:24 dist-test-slave-45dx krb5kdc[15085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977604, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:24.803309 15111 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:53:24.804134 15111 webserver.cc:460] Webserver started at http://127.15.147.190:36909/ using document root <none> and password file <none>
I20260806 00:53:24.804733 15111 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:24.804832 15111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:24.805058 15111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:24.807022 15111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "3b3e1f5d2c864b20a6783ac717920248"
format_stamp: "Formatted at 2026-08-06 00:53:24 on dist-test-slave-45dx"
server_key: "59fd7af1b33050d82627183c2d455671"
server_key_iv: "d0fc3b5c5b819142992bfa558c6aeaf5"
server_key_version: "encryptionkey@0"
I20260806 00:53:24.807578 15111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "3b3e1f5d2c864b20a6783ac717920248"
format_stamp: "Formatted at 2026-08-06 00:53:24 on dist-test-slave-45dx"
server_key: "59fd7af1b33050d82627183c2d455671"
server_key_iv: "d0fc3b5c5b819142992bfa558c6aeaf5"
server_key_version: "encryptionkey@0"
I20260806 00:53:24.811219 15111 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:53:24.813596 15130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:24.814793 15111 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 00:53:24.814939 15111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "3b3e1f5d2c864b20a6783ac717920248"
format_stamp: "Formatted at 2026-08-06 00:53:24 on dist-test-slave-45dx"
server_key: "59fd7af1b33050d82627183c2d455671"
server_key_iv: "d0fc3b5c5b819142992bfa558c6aeaf5"
server_key_version: "encryptionkey@0"
I20260806 00:53:24.815038 15111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:24.835206 15111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:24.838570 15111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:24.838778 15111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:24.846251 15111 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:41735
I20260806 00:53:24.846352 15197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:41735 every 8 connection(s)
I20260806 00:53:24.847227 15111 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:24.850111 15200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:24.853822 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 15111
I20260806 00:53:24.853966 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:24.854312 15950 external_mini_cluster.cc:1428] Setting key 73d750db991a7af20c0d3216076f7c5b
I20260806 00:53:24.856266 15200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248: Bootstrap starting.
I20260806 00:53:24.859004 15200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:24.859885 15200 log.cc:824] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:24.862087 15200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248: No bootstrap required, opened a new log
Aug 06 00:53:24 dist-test-slave-45dx krb5kdc[15085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:53:24.866292 15200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3e1f5d2c864b20a6783ac717920248" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41735 } }
I20260806 00:53:24.866568 15200 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:24.866638 15200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b3e1f5d2c864b20a6783ac717920248, State: Initialized, Role: FOLLOWER
I20260806 00:53:24.867113 15200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3e1f5d2c864b20a6783ac717920248" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41735 } }
I20260806 00:53:24.867333 15200 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:24.867409 15200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:24.867516 15200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:24.868459 15200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3e1f5d2c864b20a6783ac717920248" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41735 } }
I20260806 00:53:24.868825 15200 leader_election.cc:302] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b3e1f5d2c864b20a6783ac717920248; no voters:
I20260806 00:53:24.869174 15200 leader_election.cc:288] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:24.869272 15207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:24.869475 15207 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248 [term 1 LEADER]: Becoming Leader. State: Replica: 3b3e1f5d2c864b20a6783ac717920248, State: Running, Role: LEADER
I20260806 00:53:24.869730 15203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:24.855649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58031 (local address 127.15.147.190:41735)
0806 00:53:24.856039 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:24.856079 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:24.856118 (+ 39us) server_negotiation.cc:407] Connection header received
0806 00:53:24.856958 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:24.856982 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:24.857403 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:24.857762 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:24.858458 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:24.859996 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:24.860938 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:24.861628 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:24.861815 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:24.864295 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:24.864321 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:24.864342 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:24.864378 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:24.866399 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:24.866939 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:24.866944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:24.866949 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:24.867087 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:24.867405 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:24.867429 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:24.867431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:24.867907 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:24.868131 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:24.868431 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 00:53:24.868754 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":92,"threads_started":1}
I20260806 00:53:24.869837 15207 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3e1f5d2c864b20a6783ac717920248" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41735 } }
I20260806 00:53:24.870469 15200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:24.870998 15207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b3e1f5d2c864b20a6783ac717920248" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3e1f5d2c864b20a6783ac717920248" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41735 } } }
I20260806 00:53:24.871179 15207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:24.871613 15219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:24.872234 15208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b3e1f5d2c864b20a6783ac717920248. Latest consensus state: current_term: 1 leader_uuid: "3b3e1f5d2c864b20a6783ac717920248" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3e1f5d2c864b20a6783ac717920248" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 41735 } } }
I20260806 00:53:24.872433 15208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:24.872900 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:24.875581 15219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:24.881332 15219 catalog_manager.cc:1347] Generated new cluster ID: a91729b54770419881a48796b3bd95f8
I20260806 00:53:24.881443 15219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:24.885154 15203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:24.876535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37489 (local address 127.15.147.190:41735)
0806 00:53:24.876682 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:24.876686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:24.877061 (+ 375us) server_negotiation.cc:407] Connection header received
0806 00:53:24.877174 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:24.877177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:24.877217 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:24.877307 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:24.878089 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:24.878890 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:24.879882 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:24.880248 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:24.880423 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:24.882023 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:24.882027 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:24.882029 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:24.882058 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:24.883501 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:24.883924 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:24.883928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:24.883930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:24.884020 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:24.884298 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:24.884321 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:24.884325 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:24.884501 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:24.884637 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:24.884869 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 00:53:24.884994 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 00:53:24.906787 15219 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:24.907714 15219 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:24.914080 15219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b3e1f5d2c864b20a6783ac717920248: Generated new TSK 0
I20260806 00:53:24.914772 15219 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:24.920656 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 15111
2026-08-06T00:53:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1594 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:24 dist-test-slave-45dx krb5kdc[15248](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:24 dist-test-slave-45dx krb5kdc[15248](info): set up 2 sockets
Aug 06 00:53:24 dist-test-slave-45dx krb5kdc[15248](info): commencing operation
krb5kdc: starting...
W20260806 00:53:26.133747 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:26 dist-test-slave-45dx krb5kdc[15248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:53:26.263818 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:34153
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:34153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:26.377694 15265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:26.377957 15265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:26.378003 15265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:26.382128 15265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:26.382200 15265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:26.382218 15265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:26.382282 15265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:26.382313 15265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:26.387485 15265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:34153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:34153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:26.388581 15265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:26.389633 15265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:26.398272 15279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:26.398308 15277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:26.398307 15265 server_base.cc:1034] running on GCE node
W20260806 00:53:26.398272 15276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:26.399164 15265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:26.400259 15265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:26.401523 15265 hybrid_clock.cc:648] HybridClock initialized: now 1785977606401486 us; error 51 us; skew 500 ppm
Aug 06 00:53:26 dist-test-slave-45dx krb5kdc[15248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977606, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:26.404503 15265 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:53:26.405450 15265 webserver.cc:460] Webserver started at http://127.15.147.190:37071/ using document root <none> and password file <none>
I20260806 00:53:26.406014 15265 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:26.406076 15265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:26.406322 15265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:26.408396 15265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "9b05f3b6ae094706aaa523ece192805d"
format_stamp: "Formatted at 2026-08-06 00:53:26 on dist-test-slave-45dx"
server_key: "e423e9f7f0a017541f1d6ed83d942f0a"
server_key_iv: "554d6c2ccf4f57aa7972730b0f72f505"
server_key_version: "encryptionkey@0"
I20260806 00:53:26.408964 15265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "9b05f3b6ae094706aaa523ece192805d"
format_stamp: "Formatted at 2026-08-06 00:53:26 on dist-test-slave-45dx"
server_key: "e423e9f7f0a017541f1d6ed83d942f0a"
server_key_iv: "554d6c2ccf4f57aa7972730b0f72f505"
server_key_version: "encryptionkey@0"
I20260806 00:53:26.412587 15265 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 00:53:26.414983 15292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:26.416281 15265 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:53:26.416486 15265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "9b05f3b6ae094706aaa523ece192805d"
format_stamp: "Formatted at 2026-08-06 00:53:26 on dist-test-slave-45dx"
server_key: "e423e9f7f0a017541f1d6ed83d942f0a"
server_key_iv: "554d6c2ccf4f57aa7972730b0f72f505"
server_key_version: "encryptionkey@0"
I20260806 00:53:26.416599 15265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:26.435240 15265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:26.438489 15265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:26.438699 15265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:26.445901 15265 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:34153
I20260806 00:53:26.445928 15378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:34153 every 8 connection(s)
I20260806 00:53:26.446762 15265 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:26.449508 15381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:26.451290 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 15265
I20260806 00:53:26.451436 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:26.451711 15950 external_mini_cluster.cc:1428] Setting key ce09c3ddda8a3d7e353744f217be0520
I20260806 00:53:26.455988 15381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d: Bootstrap starting.
I20260806 00:53:26.458469 15381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:26.459414 15381 log.cc:824] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:26.460983 15386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:26.453052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58927 (local address 127.15.147.190:34153)
0806 00:53:26.453459 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:26.453478 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:26.453517 (+ 39us) server_negotiation.cc:407] Connection header received
0806 00:53:26.454255 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:26.454278 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:26.454694 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:26.455018 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:26.455390 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:26.456723 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:26.457520 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:26.458170 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:26.458333 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:26.459110 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:26.459116 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:26.459143 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:26.459631 (+ 488us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:26.459693 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:26.459730 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:26.459836 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:26.460007 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 00:53:26.460203 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":112,"threads_started":1}
I20260806 00:53:26.461495 15381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d: No bootstrap required, opened a new log
I20260806 00:53:26.463651 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:26.465586 15381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b05f3b6ae094706aaa523ece192805d" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34153 } }
I20260806 00:53:26.465906 15381 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:26.466022 15381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b05f3b6ae094706aaa523ece192805d, State: Initialized, Role: FOLLOWER
I20260806 00:53:26.466593 15381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b05f3b6ae094706aaa523ece192805d" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34153 } }
I20260806 00:53:26.466799 15381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:26.466907 15381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:26.467130 15381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:26.468281 15381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b05f3b6ae094706aaa523ece192805d" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34153 } }
I20260806 00:53:26.468662 15381 leader_election.cc:302] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b05f3b6ae094706aaa523ece192805d; no voters:
I20260806 00:53:26.469014 15381 leader_election.cc:288] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:26.469147 15396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:26.469445 15396 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d [term 1 LEADER]: Becoming Leader. State: Replica: 9b05f3b6ae094706aaa523ece192805d, State: Running, Role: LEADER
I20260806 00:53:26.470068 15396 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b05f3b6ae094706aaa523ece192805d" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34153 } }
I20260806 00:53:26.470415 15381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:26.470652 15386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:26.466019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34571 (local address 127.15.147.190:34153)
0806 00:53:26.466138 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:26.466142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:26.466375 (+ 233us) server_negotiation.cc:407] Connection header received
0806 00:53:26.466442 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:26.466444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:26.466477 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:26.466540 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:26.466921 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:26.467672 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:26.468811 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:26.469160 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:26.469287 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:26.470150 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:26.470154 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:26.470157 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:26.470170 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:26.470198 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:26.470210 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:26.470284 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:26.470455 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 00:53:26.470534 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 00:53:26.472012 15397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b05f3b6ae094706aaa523ece192805d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b05f3b6ae094706aaa523ece192805d" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34153 } } }
I20260806 00:53:26.472143 15397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:26.472541 15414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:26.472549 15399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b05f3b6ae094706aaa523ece192805d. Latest consensus state: current_term: 1 leader_uuid: "9b05f3b6ae094706aaa523ece192805d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b05f3b6ae094706aaa523ece192805d" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34153 } } }
I20260806 00:53:26.472769 15399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:26.475703 15414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:26.481068 15414 catalog_manager.cc:1347] Generated new cluster ID: d4a27510c46d459b89d7f8120fb43b2c
I20260806 00:53:26.481134 15414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:26.493747 15414 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:26.494794 15414 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:26.503134 15414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b05f3b6ae094706aaa523ece192805d: Generated new TSK 0
I20260806 00:53:26.503746 15414 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:26.514067 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 15265
2026-08-06T00:53:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1593 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:26 dist-test-slave-45dx krb5kdc[15431](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:26 dist-test-slave-45dx krb5kdc[15431](info): set up 2 sockets
Aug 06 00:53:26 dist-test-slave-45dx krb5kdc[15431](info): commencing operation
krb5kdc: starting...
W20260806 00:53:27.719336 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:27 dist-test-slave-45dx krb5kdc[15431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:53:27.840989 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:33949
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:37809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:33949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:27.952020 15451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:27.952289 15451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:27.952417 15451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:27.956207 15451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:27.956266 15451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:27.956285 15451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:27.956357 15451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:27.956388 15451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:27.961666 15451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:37809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:33949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:33949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:27.962771 15451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:27.963792 15451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:27.972563 15464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:27.972654 15451 server_base.cc:1034] running on GCE node
W20260806 00:53:27.972577 15462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:27.972563 15461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:27.973430 15451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:27.974510 15451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:27.975658 15451 hybrid_clock.cc:648] HybridClock initialized: now 1785977607975640 us; error 30 us; skew 500 ppm
Aug 06 00:53:27 dist-test-slave-45dx krb5kdc[15431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977607, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:27.982602 15451 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:53:27.983559 15451 webserver.cc:460] Webserver started at http://127.15.147.190:40077/ using document root <none> and password file <none>
I20260806 00:53:27.984164 15451 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:27.984256 15451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:27.984593 15451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:27.987231 15451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "2fcd6f7e7f3d46fe89f85c25a5d196c1"
format_stamp: "Formatted at 2026-08-06 00:53:27 on dist-test-slave-45dx"
server_key: "74a4b7a2436e2b4bec4b789c8b5a50ab"
server_key_iv: "46427ef3d2339dbfaa55afe4512a33b5"
server_key_version: "encryptionkey@0"
I20260806 00:53:27.987937 15451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "2fcd6f7e7f3d46fe89f85c25a5d196c1"
format_stamp: "Formatted at 2026-08-06 00:53:27 on dist-test-slave-45dx"
server_key: "74a4b7a2436e2b4bec4b789c8b5a50ab"
server_key_iv: "46427ef3d2339dbfaa55afe4512a33b5"
server_key_version: "encryptionkey@0"
I20260806 00:53:27.991622 15451 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 00:53:27.994026 15474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:27.995270 15451 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 00:53:27.995416 15451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "2fcd6f7e7f3d46fe89f85c25a5d196c1"
format_stamp: "Formatted at 2026-08-06 00:53:27 on dist-test-slave-45dx"
server_key: "74a4b7a2436e2b4bec4b789c8b5a50ab"
server_key_iv: "46427ef3d2339dbfaa55afe4512a33b5"
server_key_version: "encryptionkey@0"
I20260806 00:53:27.995532 15451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:28.000154 15451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:28.003139 15451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:28.003328 15451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:28.010722 15451 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:33949
I20260806 00:53:28.010746 15556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:33949 every 8 connection(s)
I20260806 00:53:28.011662 15451 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:28.014307 15557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:28.017411 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 15451
I20260806 00:53:28.017556 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:28.017822 15950 external_mini_cluster.cc:1428] Setting key 5e8e9d8869440161c66152b6a1707a81
I20260806 00:53:28.020157 15557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1: Bootstrap starting.
I20260806 00:53:28.022622 15557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:28.023058 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:28.019197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43285 (local address 127.15.147.190:33949)
0806 00:53:28.019642 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:28.019657 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:28.019690 (+ 33us) server_negotiation.cc:407] Connection header received
0806 00:53:28.020370 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:28.020387 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:28.020645 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:28.020988 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:28.021834 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:28.021839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:28.021871 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:28.021882 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:28.021936 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:28.021956 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:28.022074 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:28.022181 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 00:53:28.022336 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":137,"threads_started":1}
I20260806 00:53:28.023437 15557 log.cc:824] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:28.025532 15557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1: No bootstrap required, opened a new log
I20260806 00:53:28.025820 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:28.029637 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:28.027830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45365 (local address 127.15.147.190:33949)
0806 00:53:28.027954 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:28.027958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:28.028202 (+ 244us) server_negotiation.cc:407] Connection header received
0806 00:53:28.028384 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:28.028386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:28.028415 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:28.028478 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:28.029193 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:28.029196 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:28.029197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 00:53:28.029207 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:28.029231 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 00:53:28.029241 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:28.029320 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:28.029454 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 00:53:28.029484 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 00:53:28.029681 15557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fcd6f7e7f3d46fe89f85c25a5d196c1" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33949 } }
I20260806 00:53:28.029980 15557 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 00:53:28.030041 15557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fcd6f7e7f3d46fe89f85c25a5d196c1, State: Initialized, Role: FOLLOWER
I20260806 00:53:28.030452 15557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fcd6f7e7f3d46fe89f85c25a5d196c1" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33949 } }
I20260806 00:53:28.030644 15557 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 00:53:28.030720 15557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 00:53:28.030808 15557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:28.031620 15557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fcd6f7e7f3d46fe89f85c25a5d196c1" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33949 } }
I20260806 00:53:28.031909 15557 leader_election.cc:302] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fcd6f7e7f3d46fe89f85c25a5d196c1; no voters:
I20260806 00:53:28.032155 15557 leader_election.cc:288] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 00:53:28.032265 15570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:28.032586 15570 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1 [term 1 LEADER]: Becoming Leader. State: Replica: 2fcd6f7e7f3d46fe89f85c25a5d196c1, State: Running, Role: LEADER
I20260806 00:53:28.032935 15570 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fcd6f7e7f3d46fe89f85c25a5d196c1" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33949 } }
I20260806 00:53:28.033396 15557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:28.034600 15572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fcd6f7e7f3d46fe89f85c25a5d196c1. Latest consensus state: current_term: 1 leader_uuid: "2fcd6f7e7f3d46fe89f85c25a5d196c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fcd6f7e7f3d46fe89f85c25a5d196c1" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33949 } } }
I20260806 00:53:28.034740 15572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:28.034634 15571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fcd6f7e7f3d46fe89f85c25a5d196c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fcd6f7e7f3d46fe89f85c25a5d196c1" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 33949 } } }
I20260806 00:53:28.034804 15571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:28.035154 15581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:28.038285 15581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:28.043867 15581 catalog_manager.cc:1347] Generated new cluster ID: 076230b2e7d84966a049a328d90ba345
I20260806 00:53:28.043985 15581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:28.081235 15581 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:28.081601 15581 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:28.092005 15581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fcd6f7e7f3d46fe89f85c25a5d196c1: Generated new TSK 0
I20260806 00:53:28.092808 15581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 00:53:28.097524 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 15451
2026-08-06T00:53:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1579 ms)
[----------] 18 tests from AuthTokenIssuingTest (28805 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:28 dist-test-slave-45dx krb5kdc[15596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:28 dist-test-slave-45dx krb5kdc[15596](info): set up 2 sockets
Aug 06 00:53:28 dist-test-slave-45dx krb5kdc[15596](info): commencing operation
krb5kdc: starting...
W20260806 00:53:29.285727 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:29 dist-test-slave-45dx krb5kdc[15596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:53:29.414806 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:44295
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:39145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:44295,127.15.147.189:34599,127.15.147.188:40733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:29.524734 15614 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 00:53:29.525028 15614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:29.525096 15614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:29.525179 15614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:29.529076 15614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:29.529150 15614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:29.529187 15614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:29.529253 15614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:29.529290 15614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:29.534399 15614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:39145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:44295,127.15.147.189:34599,127.15.147.188:40733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:44295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:29.535390 15614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:29.536351 15614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:29.544819 15621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:29.544932 15623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:29.544922 15614 server_base.cc:1034] running on GCE node
W20260806 00:53:29.544862 15619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:29.545729 15614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:29.546865 15614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:29.548056 15614 hybrid_clock.cc:648] HybridClock initialized: now 1785977609548029 us; error 39 us; skew 500 ppm
Aug 06 00:53:29 dist-test-slave-45dx krb5kdc[15596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:29.550947 15614 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:53:29.551749 15614 webserver.cc:460] Webserver started at http://127.15.147.190:46435/ using document root <none> and password file <none>
I20260806 00:53:29.552237 15614 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:29.552294 15614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:29.552539 15614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:29.554522 15614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "c65c8baa02404f2aa73020c52dd2023e"
format_stamp: "Formatted at 2026-08-06 00:53:29 on dist-test-slave-45dx"
server_key: "1c1ef1a2521e29ec000d4284aef58149"
server_key_iv: "58286ac04dad3fc71cef5eb3155ff418"
server_key_version: "encryptionkey@0"
I20260806 00:53:29.555061 15614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "c65c8baa02404f2aa73020c52dd2023e"
format_stamp: "Formatted at 2026-08-06 00:53:29 on dist-test-slave-45dx"
server_key: "1c1ef1a2521e29ec000d4284aef58149"
server_key_iv: "58286ac04dad3fc71cef5eb3155ff418"
server_key_version: "encryptionkey@0"
I20260806 00:53:29.558539 15614 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260806 00:53:29.560792 15631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:29.561954 15614 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:53:29.562062 15614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "c65c8baa02404f2aa73020c52dd2023e"
format_stamp: "Formatted at 2026-08-06 00:53:29 on dist-test-slave-45dx"
server_key: "1c1ef1a2521e29ec000d4284aef58149"
server_key_iv: "58286ac04dad3fc71cef5eb3155ff418"
server_key_version: "encryptionkey@0"
I20260806 00:53:29.562144 15614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:29.587291 15614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:29.590688 15614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:29.590927 15614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:29.598119 15614 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:44295
I20260806 00:53:29.598438 15707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:44295 every 8 connection(s)
I20260806 00:53:29.599188 15614 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:29.601220 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 15614
I20260806 00:53:29.601377 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:29.601670 15950 external_mini_cluster.cc:1428] Setting key 3634db88783403c62a2768ae84dfab63
I20260806 00:53:29.602267 15709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:29.607736 15709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.147.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.189@KRBTEST.COM" created.
Aug 06 00:53:29 dist-test-slave-45dx krb5kdc[15596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:53:29.623811 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.610221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:44295 (local address 127.0.0.1:45302)
0806 00:53:29.610788 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.610836 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:29.611683 (+ 847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:29.612397 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:29.612402 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:29.612852 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:29.613187 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:29.613202 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:29.614454 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:29.614457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:29.615541 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:29.615551 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:29.616170 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:29.616174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:29.616261 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:29.616962 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:29.616978 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:29.618475 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:29.620988 (+ 2513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:29.620997 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:29.621004 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:29.621220 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:29.621642 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:29.621644 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:29.621645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:29.621751 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:29.622512 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:29.622516 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:29.622633 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 00:53:29.622766 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 00:53:29.623023 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"spinlock_wait_cycles":16640,"thread_start_us":146,"threads_started":1}
I20260806 00:53:29.623818 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.610220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45302 (local address 127.15.147.190:44295)
0806 00:53:29.610638 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:29.610657 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:29.610940 (+ 283us) server_negotiation.cc:407] Connection header received
0806 00:53:29.612018 (+ 1078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:29.612024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:29.612203 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:29.612398 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:29.613349 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:29.614317 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:29.615660 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:29.616019 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:29.616185 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:29.618641 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:29.618651 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:29.618668 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:29.618701 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:29.620875 (+ 2174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:29.621338 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:29.621346 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:29.621360 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:29.621514 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:29.621861 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:29.621864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:29.621866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:29.622376 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:29.622538 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:29.622897 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 00:53:29.623105 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":271,"thread_start_us":134,"threads_started":1}
I20260806 00:53:29.626329 15709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:53:29.627100 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.626856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:34599 (local address 127.0.0.1:40672)
0806 00:53:29.626979 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.627046 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 00:53:29.627316 15636 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34599: connect: Connection refused (error 111)
I20260806 00:53:29.628335 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.628081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:34599 (local address 127.0.0.1:40688)
0806 00:53:29.628192 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.628254 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 00:53:29.628492 15709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.189:34599: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34599: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.189.keytab.
Entry for principal kudu/127.15.147.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.189.keytab.
I20260806 00:53:29.632706 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.189:34599
--webserver_interface=127.15.147.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.189.keytab
--principal=kudu/127.15.147.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:39145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:44295,127.15.147.189:34599,127.15.147.188:40733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:53:29.677762 15709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } attempt: 1
I20260806 00:53:29.678702 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.678387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:34599 (local address 127.0.0.1:40700)
0806 00:53:29.678552 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.678620 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 00:53:29.679457 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.679236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:34599 (local address 127.0.0.1:40712)
0806 00:53:29.679349 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.679401 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 00:53:29.679584 15709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.189:34599: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34599: connect: Connection refused (error 111)
W20260806 00:53:29.743706 15717 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 00:53:29.744033 15717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:29.744108 15717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:29.744181 15717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 00:53:29.747815 15709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } attempt: 2
W20260806 00:53:29.748397 15717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:29.748474 15717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:29.748539 15717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:29.748579 15717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:29.748617 15717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:29.748718 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.748370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:34599 (local address 127.0.0.1:40726)
0806 00:53:29.748509 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.748583 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 00:53:29.749486 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.749257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:34599 (local address 127.0.0.1:40740)
0806 00:53:29.749368 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.749433 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 00:53:29.749662 15709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.189:34599: Network error: Client connection negotiation failed: client connection to 127.15.147.189:34599: connect: Connection refused (error 111)
I20260806 00:53:29.753931 15717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:39145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:44295,127.15.147.189:34599,127.15.147.188:40733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.189.keytab
--principal=kudu/127.15.147.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.189:34599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.147.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:29.754832 15717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:29.755718 15717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:29.764032 15724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:29.764045 15727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:29.764045 15725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:29.764338 15717 server_base.cc:1034] running on GCE node
I20260806 00:53:29.764945 15717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:29.766026 15717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:29.767216 15717 hybrid_clock.cc:648] HybridClock initialized: now 1785977609767194 us; error 41 us; skew 500 ppm
Aug 06 00:53:29 dist-test-slave-45dx krb5kdc[15596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:29.770057 15717 init.cc:377] Logged in from keytab as kudu/127.15.147.189@KRBTEST.COM (short username kudu)
I20260806 00:53:29.771031 15717 webserver.cc:460] Webserver started at http://127.15.147.189:38597/ using document root <none> and password file <none>
I20260806 00:53:29.771574 15717 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:29.771663 15717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:29.771893 15717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:29.773871 15717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-1/data/instance:
uuid: "e34b3f6de27a475f9f99daee81029fd7"
format_stamp: "Formatted at 2026-08-06 00:53:29 on dist-test-slave-45dx"
server_key: "41f532ec0303d0957cee5959329edb45"
server_key_iv: "50d1d114a4ccaac47c84f007f6ba32f0"
server_key_version: "encryptionkey@0"
I20260806 00:53:29.774443 15717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-1/wal/instance:
uuid: "e34b3f6de27a475f9f99daee81029fd7"
format_stamp: "Formatted at 2026-08-06 00:53:29 on dist-test-slave-45dx"
server_key: "41f532ec0303d0957cee5959329edb45"
server_key_iv: "50d1d114a4ccaac47c84f007f6ba32f0"
server_key_version: "encryptionkey@0"
I20260806 00:53:29.777947 15717 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260806 00:53:29.780268 15736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:29.781533 15717 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 00:53:29.781669 15717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-1/wal
uuid: "e34b3f6de27a475f9f99daee81029fd7"
format_stamp: "Formatted at 2026-08-06 00:53:29 on dist-test-slave-45dx"
server_key: "41f532ec0303d0957cee5959329edb45"
server_key_iv: "50d1d114a4ccaac47c84f007f6ba32f0"
server_key_version: "encryptionkey@0"
I20260806 00:53:29.781800 15717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:29.799765 15717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:29.802594 15717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:29.802795 15717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:29.810111 15717 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.189:34599
I20260806 00:53:29.810374 15808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.189:34599 every 8 connection(s)
I20260806 00:53:29.811044 15717 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-1/data/info.pb
I20260806 00:53:29.813680 15809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:29.817164 15809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:53:29.819953 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 15717
I20260806 00:53:29.820088 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-1/wal/instance
I20260806 00:53:29.820429 15950 external_mini_cluster.cc:1428] Setting key 6bdf18c62929fabf56c4737318b4f16f
Aug 06 00:53:29 dist-test-slave-45dx krb5kdc[15596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.189@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.147.188@KRBTEST.COM; defaulting to no policy
I20260806 00:53:29.831429 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.818891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45314 (local address 127.15.147.190:44295)
0806 00:53:29.819058 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:29.819063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:29.819836 (+ 773us) server_negotiation.cc:407] Connection header received
0806 00:53:29.821058 (+ 1222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:29.821061 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:29.821112 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:29.821178 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:29.822779 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:29.823599 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:29.824730 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:29.825150 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:29.825273 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:29.827473 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:29.827477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:29.827480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:29.827511 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:29.829261 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:29.829809 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:29.829815 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:29.829818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:29.829918 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:29.830254 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:29.830259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:29.830264 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:29.830525 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:29.830636 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:29.831147 (+ 511us) server_negotiation.cc:299] Negotiation successful
0806 00:53:29.831292 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 00:53:29.832191 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.819244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:44295 (local address 127.0.0.1:45314)
0806 00:53:29.819660 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.819721 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:29.820815 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:29.821393 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:29.821404 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:29.821916 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:29.822587 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:29.822609 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:29.823740 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:29.823743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:29.824603 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:29.824613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:29.825288 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:29.825290 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:29.825411 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:29.825969 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:29.825983 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:29.827275 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:29.829384 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:29.829389 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:29.829399 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:29.829666 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:29.830022 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:29.830024 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:29.830026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:29.830151 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:29.830652 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:29.830656 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:29.830872 (+ 216us) client_negotiation.cc:769] Sending connection context
0806 00:53:29.831047 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 00:53:29.831348 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":100,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.188@KRBTEST.COM" created.
I20260806 00:53:29.833762 15809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:53:29 dist-test-slave-45dx krb5kdc[15596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.189@KRBTEST.COM for kudu/127.15.147.189@KRBTEST.COM
I20260806 00:53:29.840847 15709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } attempt: 3
I20260806 00:53:29.843344 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.834268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:34599 (local address 127.0.0.1:40744)
0806 00:53:29.834396 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.834410 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:29.834481 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:29.835105 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:29.835108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:29.835282 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:29.835362 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:29.835367 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:29.836358 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:29.836361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:29.836886 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:29.836893 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:29.837487 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:29.837491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:29.837641 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:29.838144 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:29.838156 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:29.839393 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:29.841651 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:29.841654 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:29.841656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:29.841952 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:29.842301 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:29.842304 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:29.842306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:29.842409 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:29.842863 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:29.842869 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:29.842989 (+ 120us) client_negotiation.cc:769] Sending connection context
0806 00:53:29.843092 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 00:53:29.843224 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":4}
I20260806 00:53:29.844198 15815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.834411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40744 (local address 127.15.147.189:34599)
0806 00:53:29.834764 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:29.834772 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:29.834790 (+ 18us) server_negotiation.cc:407] Connection header received
0806 00:53:29.834822 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:29.834826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:29.834984 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:29.835125 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:29.835479 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:29.836215 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:29.837000 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:29.837363 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:29.837482 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:29.839541 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:29.839554 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:29.839568 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:29.839599 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:29.841525 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:29.842079 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:29.842086 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:29.842088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:29.842185 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:29.842513 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:29.842516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:29.842517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:29.842750 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:29.842856 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:29.843612 (+ 756us) server_negotiation.cc:299] Negotiation successful
0806 00:53:29.843789 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":268,"thread_start_us":163,"threads_started":1}
I20260806 00:53:29.846180 15809 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:53:29.846879 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.846623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:40733 (local address 127.0.0.1:33238)
0806 00:53:29.846719 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.846803 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
Aug 06 00:53:29 dist-test-slave-45dx krb5kdc[15596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for kudu/127.15.147.189@KRBTEST.COM
W20260806 00:53:29.847086 15739 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
I20260806 00:53:29.848012 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.847800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:40733 (local address 127.0.0.1:33242)
0806 00:53:29.847905 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.847953 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 00:53:29.848199 15809 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.188:40733: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.188.keytab.
Entry for principal kudu/127.15.147.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.188.keytab.
I20260806 00:53:29.851105 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.841324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:34599 (local address 127.0.0.1:40748)
0806 00:53:29.841536 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.841551 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:29.841659 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:29.842184 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:29.842188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:29.842389 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:29.842963 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:29.842974 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:29.843928 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:29.843932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:29.844789 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:29.844800 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:29.845362 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:29.845365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:29.845444 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:29.845938 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:29.845953 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:29.847493 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:29.849659 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:29.849661 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:29.849663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:29.849848 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:29.850191 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:29.850197 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:29.850202 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:29.850305 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:29.850730 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:29.850733 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:29.850813 (+ 80us) client_negotiation.cc:769] Sending connection context
0806 00:53:29.850899 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 00:53:29.850997 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260806 00:53:29.851135 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.188:40733
--webserver_interface=127.15.147.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.188.keytab
--principal=kudu/127.15.147.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:39145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:44295,127.15.147.189:34599,127.15.147.188:40733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:53:29.851275 15817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.841365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40748 (local address 127.15.147.189:34599)
0806 00:53:29.841941 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:29.841951 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:29.841964 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:53:29.841998 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:29.842004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:29.842051 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:29.842155 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:29.843103 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:29.843815 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:29.844909 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:29.845237 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:29.845333 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:29.847629 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:29.847633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:29.847635 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:29.847658 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:29.849563 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:29.849981 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:29.849984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:29.849986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:29.850073 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:29.850404 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:29.850407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:29.850408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:29.850630 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:29.850731 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:29.850995 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 00:53:29.851101 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":110,"threads_started":1}
I20260806 00:53:29.851915 15709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:53:29.852720 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.852428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:40733 (local address 127.0.0.1:33254)
0806 00:53:29.852570 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.852635 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 00:53:29.853500 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.853213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:40733 (local address 127.0.0.1:33258)
0806 00:53:29.853368 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.853428 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260806 00:53:29.853642 15709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.188:40733: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
I20260806 00:53:29.884785 15709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } attempt: 1
I20260806 00:53:29.885746 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.885414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:40733 (local address 127.0.0.1:33262)
0806 00:53:29.885580 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.885654 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 00:53:29.886504 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.886298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:40733 (local address 127.0.0.1:33270)
0806 00:53:29.886403 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.886445 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 00:53:29.886708 15709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.188:40733: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
I20260806 00:53:29.897471 15809 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } attempt: 1
I20260806 00:53:29.898170 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.897906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:40733 (local address 127.0.0.1:33278)
0806 00:53:29.898047 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.898104 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 00:53:29.898886 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.898678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:40733 (local address 127.0.0.1:33294)
0806 00:53:29.898786 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.898836 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 00:53:29.899019 15809 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.188:40733: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
I20260806 00:53:29.961884 15709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } attempt: 2
I20260806 00:53:29.962746 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.962452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:40733 (local address 127.0.0.1:33306)
0806 00:53:29.962604 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.962668 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 00:53:29.963497 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.963297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:40733 (local address 127.0.0.1:33308)
0806 00:53:29.963394 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.963437 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 00:53:29.963661 15709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.188:40733: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
W20260806 00:53:29.964991 15819 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 00:53:29.965260 15819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:29.965302 15819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:29.965377 15819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 00:53:29.967202 15809 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } attempt: 2
I20260806 00:53:29.967870 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.967591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:40733 (local address 127.0.0.1:33314)
0806 00:53:29.967749 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.967806 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 00:53:29.968596 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:29.968388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:40733 (local address 127.0.0.1:33320)
0806 00:53:29.968488 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:29.968538 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 00:53:29.968720 15809 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.188:40733: Network error: Client connection negotiation failed: client connection to 127.15.147.188:40733: connect: Connection refused (error 111)
W20260806 00:53:29.969679 15819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:29.969726 15819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:29.969745 15819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:29.969761 15819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:29.969776 15819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:29.974807 15819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:39145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:44295,127.15.147.189:34599,127.15.147.188:40733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.188.keytab
--principal=kudu/127.15.147.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.188:40733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.147.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:29.975723 15819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:29.976704 15819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:29.985087 15834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:29.985081 15832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:29.985261 15819 server_base.cc:1034] running on GCE node
W20260806 00:53:29.985383 15828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:29.986089 15819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:29.987114 15819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:29.988355 15819 hybrid_clock.cc:648] HybridClock initialized: now 1785977609988337 us; error 27 us; skew 500 ppm
Aug 06 00:53:29 dist-test-slave-45dx krb5kdc[15596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:29.991209 15819 init.cc:377] Logged in from keytab as kudu/127.15.147.188@KRBTEST.COM (short username kudu)
I20260806 00:53:29.992308 15819 webserver.cc:460] Webserver started at http://127.15.147.188:34581/ using document root <none> and password file <none>
I20260806 00:53:29.992995 15819 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:29.993095 15819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:29.993335 15819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:29.995831 15819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-2/data/instance:
uuid: "5f203d0b59e1420989f18e6d93c452c9"
format_stamp: "Formatted at 2026-08-06 00:53:29 on dist-test-slave-45dx"
server_key: "a1b94dfeae734cc6ac62e3cd00f6515b"
server_key_iv: "772abd8d99a77b55bd4a147a54143bb1"
server_key_version: "encryptionkey@0"
I20260806 00:53:29.996390 15819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-2/wal/instance:
uuid: "5f203d0b59e1420989f18e6d93c452c9"
format_stamp: "Formatted at 2026-08-06 00:53:29 on dist-test-slave-45dx"
server_key: "a1b94dfeae734cc6ac62e3cd00f6515b"
server_key_iv: "772abd8d99a77b55bd4a147a54143bb1"
server_key_version: "encryptionkey@0"
I20260806 00:53:29.999904 15819 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 00:53:30.002451 15849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:30.003819 15819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 00:53:30.003947 15819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-2/wal
uuid: "5f203d0b59e1420989f18e6d93c452c9"
format_stamp: "Formatted at 2026-08-06 00:53:29 on dist-test-slave-45dx"
server_key: "a1b94dfeae734cc6ac62e3cd00f6515b"
server_key_iv: "772abd8d99a77b55bd4a147a54143bb1"
server_key_version: "encryptionkey@0"
I20260806 00:53:30.004045 15819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:30.018615 15819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:30.022024 15819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:30.022240 15819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:30.029305 15819 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.188:40733
I20260806 00:53:30.029613 15921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.188:40733 every 8 connection(s)
I20260806 00:53:30.030278 15819 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-2/data/info.pb
I20260806 00:53:30.032966 15924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:30.036692 15924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:53:30.038116 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 15819
I20260806 00:53:30.038271 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785977317010501-15950-0/minicluster-data/master-2/wal/instance
I20260806 00:53:30.038625 15950 external_mini_cluster.cc:1428] Setting key 8b9367d4845966ec8648c9e72adc7b71
Aug 06 00:53:30 dist-test-slave-45dx krb5kdc[15596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.188@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
Aug 06 00:53:30 dist-test-slave-45dx krb5kdc[15596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.188@KRBTEST.COM
I20260806 00:53:30.051143 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:30.038541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45322 (local address 127.15.147.190:44295)
0806 00:53:30.038694 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.038699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.039720 (+ 1021us) server_negotiation.cc:407] Connection header received
0806 00:53:30.040915 (+ 1195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.040918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.040968 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.041036 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.042073 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.042583 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.043706 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.043999 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.044082 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:30.046847 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:30.046851 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:30.046854 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:30.046895 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:30.048678 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.049387 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.049390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.049391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.049487 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.049839 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.049844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.049845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.050091 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:30.050215 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:30.050826 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 00:53:30.050950 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 00:53:30.052078 15929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:30.039301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33326 (local address 127.15.147.188:40733)
0806 00:53:30.039735 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.039748 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.039813 (+ 65us) server_negotiation.cc:407] Connection header received
0806 00:53:30.040688 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.040712 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.040963 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.041178 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.041958 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.042993 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.043896 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.044379 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.044481 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:30.047370 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:30.047376 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:30.047392 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:30.047414 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:30.049266 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.050002 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.050006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.050007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.050079 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.050403 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.050406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.050408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.050669 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:30.050860 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:30.051144 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 00:53:30.051271 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":147,"threads_started":1}
I20260806 00:53:30.052093 15926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:30.038938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:44295 (local address 127.0.0.1:45322)
0806 00:53:30.039559 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:30.039616 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:30.040688 (+ 1072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:30.041117 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:30.041122 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:30.041609 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:30.041908 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:30.041929 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.042699 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.042702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:30.043581 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:30.043589 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.044145 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.044149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:30.044350 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:30.045034 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:30.045051 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:30.046647 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:30.048789 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:30.048797 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:30.048807 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:30.049262 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:30.049584 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:30.049587 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:30.049588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:30.049719 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:30.050223 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:30.050228 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:30.050517 (+ 289us) client_negotiation.cc:769] Sending connection context
0806 00:53:30.050730 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 00:53:30.051047 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":147,"threads_started":1}
I20260806 00:53:30.053555 15924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:53:30 dist-test-slave-45dx krb5kdc[15596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.188@KRBTEST.COM for kudu/127.15.147.189@KRBTEST.COM
Aug 06 00:53:30 dist-test-slave-45dx krb5kdc[15596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
I20260806 00:53:30.059880 15809 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } attempt: 3
I20260806 00:53:30.062738 15926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:30.054058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:34599 (local address 127.0.0.1:40758)
0806 00:53:30.054165 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:30.054181 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:30.054256 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:30.054513 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:30.054516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:30.054682 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:30.054796 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:30.054803 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.055902 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.055906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:30.056638 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:30.056647 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.057243 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.057245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:30.057347 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:30.057734 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:30.057752 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:30.059113 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:30.061184 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:30.061187 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:30.061189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:30.061380 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:30.061672 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:30.061675 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:30.061676 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:30.061779 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:30.062156 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:30.062159 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:30.062251 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 00:53:30.062418 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 00:53:30.062553 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 00:53:30.062817 15709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } attempt: 3
I20260806 00:53:30.063632 15817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:30.054195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40758 (local address 127.15.147.189:34599)
0806 00:53:30.054299 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.054302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.054314 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:53:30.054355 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.054357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.054400 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.054498 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.054949 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.055790 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.056745 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.057132 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.057239 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:30.059264 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:30.059266 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:30.059268 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:30.059292 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:30.061074 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.061482 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.061487 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.061488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.061573 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.061870 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.061875 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.061880 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.062056 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:30.062184 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:30.063333 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0806 00:53:30.063471 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 00:53:30.064415 15924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 00:53:30 dist-test-slave-45dx krb5kdc[15596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.189@KRBTEST.COM for kudu/127.15.147.188@KRBTEST.COM
I20260806 00:53:30.065680 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:30.054709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45328 (local address 127.15.147.190:44295)
0806 00:53:30.054822 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.054828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.054844 (+ 16us) server_negotiation.cc:407] Connection header received
0806 00:53:30.054953 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.054955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.054990 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.055042 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.055722 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.056419 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.057351 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.057686 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.057801 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:30.060056 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:30.060059 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:30.060062 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:30.060089 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:30.062254 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.062736 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.062739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.062741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.063571 (+ 830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.063903 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.063907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.063908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.064436 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:30.064690 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:30.065419 (+ 729us) server_negotiation.cc:299] Negotiation successful
0806 00:53:30.065526 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 00:53:30 dist-test-slave-45dx krb5kdc[15596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for kudu/127.15.147.188@KRBTEST.COM
I20260806 00:53:30.070252 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:30.060336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:40733 (local address 127.0.0.1:33342)
0806 00:53:30.060477 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:30.060493 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:30.060572 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:30.061111 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:30.061115 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:30.061297 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:30.061376 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:30.061382 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.062444 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.062447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:30.063158 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:30.063170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.063754 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.063760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:30.063859 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:30.064401 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:30.064414 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:30.065850 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:30.068726 (+ 2876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:30.068731 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:30.068733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:30.068978 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:30.069292 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:30.069296 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:30.069298 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:30.069407 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:30.069801 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:30.069805 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:30.069903 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 00:53:30.069980 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 00:53:30.070102 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 00:53:30.071074 15929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:30.060719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33342 (local address 127.15.147.188:40733)
0806 00:53:30.060868 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.060872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.060889 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:53:30.060926 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.060929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.060975 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.061062 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.061499 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.062282 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.063272 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.063620 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.064052 (+ 432us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:30.065961 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:30.065964 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:30.065966 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:30.065990 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:30.068589 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.069104 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.069107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.069108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.069189 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.069515 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.069518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.069520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.069699 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:30.070727 (+ 1028us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:30.070808 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 00:53:30.070935 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 00:53:30 dist-test-slave-45dx krb5kdc[15596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.188@KRBTEST.COM for kudu/127.15.147.188@KRBTEST.COM
Aug 06 00:53:30 dist-test-slave-45dx krb5kdc[15596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.189@KRBTEST.COM
I20260806 00:53:30.075623 15809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7: Bootstrap starting.
I20260806 00:53:30.075843 15926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:30.064833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:40733 (local address 127.0.0.1:33362)
0806 00:53:30.064947 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:30.064961 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:30.065033 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:30.066484 (+ 1451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:30.066486 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:30.066660 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:30.066732 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:30.066738 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.068194 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.068198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:30.069055 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:30.069067 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.069711 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.069716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:30.069810 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:30.070243 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:30.070260 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:30.071706 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:30.074181 (+ 2475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:30.074184 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:30.074186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:30.074454 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:30.074724 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:30.074729 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:30.074733 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:30.074833 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:30.075326 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:30.075329 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:30.075418 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 00:53:30.075526 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 00:53:30.075642 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 00:53:30.076002 15933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:30.064976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33362 (local address 127.15.147.188:40733)
0806 00:53:30.066061 (+ 1085us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.066072 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.066082 (+ 10us) server_negotiation.cc:407] Connection header received
0806 00:53:30.066107 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.066108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.066137 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.066246 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.066840 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.068077 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.069191 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.069584 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.070022 (+ 438us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:30.071838 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:30.071842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:30.071844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:30.071877 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:30.074059 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.074559 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.074561 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.074563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.074636 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.074948 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.074967 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.074969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.075235 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:30.075337 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:30.075612 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 00:53:30.075730 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1034,"thread_start_us":98,"threads_started":1}
I20260806 00:53:30.078341 15924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9: Bootstrap starting.
I20260806 00:53:30.078516 15809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:30.079262 15809 log.cc:824] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:30.080613 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:30.063149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:40733 (local address 127.0.0.1:33354)
0806 00:53:30.063272 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:30.063286 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:30.063353 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:30.063861 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:30.063865 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:30.064043 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:30.064137 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:30.064146 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.065652 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.065655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:30.066453 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:30.066467 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.067056 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.067060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:30.067164 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:30.067617 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:30.067632 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:30.069087 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:30.071471 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:30.071475 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:30.071477 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:30.071724 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:30.072448 (+ 724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:30.072451 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:30.072453 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:30.072559 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:30.080045 (+ 7486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:30.080049 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:30.080149 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 00:53:30.080266 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 00:53:30.080443 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 00:53:30.080641 15930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:30.063313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33354 (local address 127.15.147.188:40733)
0806 00:53:30.063626 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.063634 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.063647 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:53:30.063680 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.063682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.063727 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.064245 (+ 518us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.064469 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.065559 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.066578 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.066933 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.067419 (+ 486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:30.069221 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:30.069224 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:30.069227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:30.069249 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:30.071323 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.072224 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.072226 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.072227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.072366 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.072674 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.079678 (+ 7004us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.079682 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.079918 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:30.080026 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:30.080368 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 00:53:30.080479 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":132,"threads_started":1}
I20260806 00:53:30.080490 15924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:30.081488 15809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7: No bootstrap required, opened a new log
I20260806 00:53:30.081686 15924 log.cc:824] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:30.083688 15709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e: Bootstrap starting.
I20260806 00:53:30.083974 15924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9: No bootstrap required, opened a new log
I20260806 00:53:30.084951 15809 raft_consensus.cc:348] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } } peers { permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } } peers { permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } }
I20260806 00:53:30.085181 15809 raft_consensus.cc:729] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e34b3f6de27a475f9f99daee81029fd7, State: Initialized, Role: FOLLOWER
I20260806 00:53:30.085474 15809 consensus_queue.cc:260] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } } peers { permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } } peers { permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } }
I20260806 00:53:30.085988 15936 sys_catalog.cc:455] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } } peers { permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } } peers { permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } } }
I20260806 00:53:30.086181 15936 sys_catalog.cc:458] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:53:30.086212 15709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:30.087000 15809 sys_catalog.cc:564] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:30.087114 15709 log.cc:824] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:30.088639 15924 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } } peers { permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } } peers { permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } }
I20260806 00:53:30.088871 15924 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f203d0b59e1420989f18e6d93c452c9, State: Initialized, Role: FOLLOWER
I20260806 00:53:30.089118 15709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e: No bootstrap required, opened a new log
I20260806 00:53:30.090034 15817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:30.067676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40772 (local address 127.15.147.189:34599)
0806 00:53:30.067774 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.067778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.067870 (+ 92us) server_negotiation.cc:407] Connection header received
0806 00:53:30.068079 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.068084 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.068127 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.068176 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.069073 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.069767 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.070782 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.071133 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.071222 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:30.074258 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:30.074261 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:30.074263 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:30.074286 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:30.076088 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.076610 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.076615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.076617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.080571 (+ 3954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.082344 (+ 1773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.082347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.082349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.082555 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:30.082745 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:30.089768 (+ 7023us) server_negotiation.cc:299] Negotiation successful
0806 00:53:30.089897 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 00:53:30.092000 15957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:53:30.092211 15957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:53:30.093035 15924 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } } peers { permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } } peers { permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } }
I20260806 00:53:30.093211 15709 raft_consensus.cc:348] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } } peers { permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } } peers { permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } }
I20260806 00:53:30.093482 15709 raft_consensus.cc:729] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c65c8baa02404f2aa73020c52dd2023e, State: Initialized, Role: FOLLOWER
I20260806 00:53:30.093878 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:30.093837 15709 consensus_queue.cc:260] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } } peers { permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } } peers { permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } }
I20260806 00:53:30.090943 15717 master_runner.cc:186] Error getting master registration for 127.15.147.190:44295: OK, instance_id { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" instance_seqno: 1785977609596012 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 00:53:30.094219 15717 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 00:53:30.094447 15924 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:30.095005 15709 sys_catalog.cc:564] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:30.095403 15675 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c65c8baa02404f2aa73020c52dd2023e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45328
I20260806 00:53:30.095678 15675 raft_consensus.cc:482] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 00:53:30.095783 15675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:30.097084 15675 raft_consensus.cc:504] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } } peers { permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } } peers { permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } }
I20260806 00:53:30.097996 15675 leader_election.cc:288] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [CANDIDATE]: Term 1 election: Requested vote from peers e34b3f6de27a475f9f99daee81029fd7 (127.15.147.189:34599), 5f203d0b59e1420989f18e6d93c452c9 (127.15.147.188:40733)
I20260806 00:53:30.098809 15885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c65c8baa02404f2aa73020c52dd2023e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5f203d0b59e1420989f18e6d93c452c9"
I20260806 00:53:30.099079 15774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c65c8baa02404f2aa73020c52dd2023e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e34b3f6de27a475f9f99daee81029fd7"
I20260806 00:53:30.099278 15774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:30.100543 15961 sys_catalog.cc:455] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } } peers { permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } } peers { permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } } }
I20260806 00:53:30.100657 15774 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c65c8baa02404f2aa73020c52dd2023e in term 1.
I20260806 00:53:30.100690 15961 sys_catalog.cc:458] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:53:30.101158 15636 leader_election.cc:302] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c65c8baa02404f2aa73020c52dd2023e, e34b3f6de27a475f9f99daee81029fd7; no voters:
I20260806 00:53:30.097921 15819 master_runner.cc:186] Error getting master registration for 127.15.147.190:44295: OK, instance_id { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" instance_seqno: 1785977609596012 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 00:53:30.101493 15819 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 00:53:30.101498 15961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:30.101720 15961 raft_consensus.cc:686] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [term 1 LEADER]: Becoming Leader. State: Replica: c65c8baa02404f2aa73020c52dd2023e, State: Running, Role: LEADER
I20260806 00:53:30.101922 15885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:30.102102 15961 consensus_queue.cc:237] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } } peers { permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } } peers { permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } }
I20260806 00:53:30.103154 15885 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c65c8baa02404f2aa73020c52dd2023e in term 1.
I20260806 00:53:30.104307 16000 sys_catalog.cc:455] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c65c8baa02404f2aa73020c52dd2023e. Latest consensus state: current_term: 1 leader_uuid: "c65c8baa02404f2aa73020c52dd2023e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } } peers { permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } } peers { permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } } }
I20260806 00:53:30.104447 16000 sys_catalog.cc:458] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:30.106300 15959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } } peers { permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } } peers { permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } } }
I20260806 00:53:30.106431 15959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 00:53:30.106832 15984 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:53:30.106938 15984 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 00:53:30.107396 15992 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:53:30.107569 15992 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:53:30.108610 16003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:30.109591 16003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:30.114616 15885 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9 [term 1 FOLLOWER]: Refusing update from remote peer c65c8baa02404f2aa73020c52dd2023e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:30.115347 15774 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7 [term 1 FOLLOWER]: Refusing update from remote peer c65c8baa02404f2aa73020c52dd2023e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:30.116130 15961 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:30.116366 16000 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:30.120046 15959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c65c8baa02404f2aa73020c52dd2023e. Latest consensus state: current_term: 1 leader_uuid: "c65c8baa02404f2aa73020c52dd2023e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } } peers { permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } } peers { permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } } }
I20260806 00:53:30.120074 15936 sys_catalog.cc:455] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c65c8baa02404f2aa73020c52dd2023e. Latest consensus state: current_term: 1 leader_uuid: "c65c8baa02404f2aa73020c52dd2023e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } } peers { permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } } peers { permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } } }
I20260806 00:53:30.120193 15936 sys_catalog.cc:458] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:53:30.120196 15959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:53:30.120443 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:30.104740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45332 (local address 127.15.147.190:44295)
0806 00:53:30.104863 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.104868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.106755 (+ 1887us) server_negotiation.cc:407] Connection header received
0806 00:53:30.106966 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.106971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.107014 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.107065 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.108225 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.108976 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.110180 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.110524 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.110601 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:30.112513 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:30.112517 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:30.112520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:30.112545 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:30.114250 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.114764 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.114768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.114770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.114858 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.115225 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.115229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.115231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.115447 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:30.115569 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:30.115956 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 00:53:30.116080 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:53:30.123307 16000 sys_catalog.cc:455] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c65c8baa02404f2aa73020c52dd2023e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } } peers { permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } } peers { permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } } }
I20260806 00:53:30.123417 16000 sys_catalog.cc:458] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:30.124402 15936 sys_catalog.cc:455] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c65c8baa02404f2aa73020c52dd2023e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } } peers { permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } } peers { permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } } }
I20260806 00:53:30.124516 15936 sys_catalog.cc:458] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:53:30.124956 16011 mvcc.cc:205] Tried to move back new op lower bound from 7315364291022299136 to 7315364290981634048. Current Snapshot: MvccSnapshot[applied={T|T < 7315364291022299136}]
I20260806 00:53:30.127104 16017 mvcc.cc:205] Tried to move back new op lower bound from 7315364291022299136 to 7315364290981634048. Current Snapshot: MvccSnapshot[applied={T|T < 7315364291022299136}]
I20260806 00:53:30.127306 15817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:30.106776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40780 (local address 127.15.147.189:34599)
0806 00:53:30.106960 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.106964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.109894 (+ 2930us) server_negotiation.cc:407] Connection header received
0806 00:53:30.116498 (+ 6604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.116502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.116549 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.116597 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.119418 (+ 2821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.120111 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.121087 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.121439 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.121544 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:30.123473 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:30.123477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:30.123480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:30.123509 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:30.125310 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.125835 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.125839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.125840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.125933 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.126266 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.126269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.126271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.126512 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:30.126618 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:30.127007 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 00:53:30.127146 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260806 00:53:30.129607 15961 sys_catalog.cc:455] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c65c8baa02404f2aa73020c52dd2023e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } } peers { permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } } peers { permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } } }
I20260806 00:53:30.129720 15961 sys_catalog.cc:458] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:30.130719 15959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c65c8baa02404f2aa73020c52dd2023e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c65c8baa02404f2aa73020c52dd2023e" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 44295 } } peers { permanent_uuid: "e34b3f6de27a475f9f99daee81029fd7" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 34599 } } peers { permanent_uuid: "5f203d0b59e1420989f18e6d93c452c9" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 40733 } } }
I20260806 00:53:30.130869 15959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:53:30.131003 16003 catalog_manager.cc:1347] Generated new cluster ID: ddc81c5a70b94a49b5ae6c0dabb98d2d
I20260806 00:53:30.131054 16003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:30.133541 15930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:30.105480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33372 (local address 127.15.147.188:40733)
0806 00:53:30.105576 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.105580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.108086 (+ 2506us) server_negotiation.cc:407] Connection header received
0806 00:53:30.108727 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.108731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.108775 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.108821 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.117545 (+ 8724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.118307 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.121111 (+ 2804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:30.121477 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:30.121548 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:30.129961 (+ 8413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:30.129964 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:30.129968 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:30.129996 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:30.131668 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.132087 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.132090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.132092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.132182 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:30.132515 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:30.132519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:30.132521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:30.132732 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:30.132858 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:30.133241 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 00:53:30.133384 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 00:53:30.168247 16003 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:30.169684 16003 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:30.192883 16003 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c65c8baa02404f2aa73020c52dd2023e: Generated new TSK 0
I20260806 00:53:30.193825 16003 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:53:30.226295 15817 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:30.223021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40792 (local address 127.15.147.189:34599)
0806 00:53:30.223163 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.223167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.225118 (+ 1951us) server_negotiation.cc:407] Connection header received
0806 00:53:30.225337 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.225342 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.225401 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.225455 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.226157 (+ 702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40792: BlockingRecv error: recv got EOF from 127.0.0.1:40792 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 00:53:30.228437 15713 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:30.224561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45348 (local address 127.15.147.190:44295)
0806 00:53:30.224666 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.224671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.227183 (+ 2512us) server_negotiation.cc:407] Connection header received
0806 00:53:30.227441 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.227445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.227493 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.227543 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.228257 (+ 714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45348: BlockingRecv error: recv got EOF from 127.0.0.1:45348 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 00:53:30.229312 15930 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:30.223845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33382 (local address 127.15.147.188:40733)
0806 00:53:30.223963 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.223967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.226867 (+ 2900us) server_negotiation.cc:407] Connection header received
0806 00:53:30.227670 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.227673 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.227717 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.227764 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.229192 (+ 1428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33382: BlockingRecv error: recv got EOF from 127.0.0.1:33382 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 00:53:30.233867 15817 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:30.232279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40804 (local address 127.15.147.189:34599)
0806 00:53:30.232396 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.232399 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.232872 (+ 473us) server_negotiation.cc:407] Connection header received
0806 00:53:30.233085 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.233088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.233162 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.233208 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.233769 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40804: BlockingRecv error: recv got EOF from 127.0.0.1:40804 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 00:53:30.238631 15817 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:30.237201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40808 (local address 127.15.147.189:34599)
0806 00:53:30.237289 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.237293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.237795 (+ 502us) server_negotiation.cc:407] Connection header received
0806 00:53:30.237964 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.237967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.238012 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.238059 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.238543 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40808: BlockingRecv error: recv got EOF from 127.0.0.1:40808 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260806 00:53:30.243813 15817 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:30.242483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40824 (local address 127.15.147.189:34599)
0806 00:53:30.242572 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.242576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.243033 (+ 457us) server_negotiation.cc:407] Connection header received
0806 00:53:30.243193 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.243196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.243237 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.243291 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.243731 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40824: BlockingRecv error: recv got EOF from 127.0.0.1:40824 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260806 00:53:30.251147 15817 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:30.249685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40826 (local address 127.15.147.189:34599)
0806 00:53:30.249808 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.249812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.250293 (+ 481us) server_negotiation.cc:407] Connection header received
0806 00:53:30.250492 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.250495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.250542 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.250589 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.251061 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40826: BlockingRecv error: recv got EOF from 127.0.0.1:40826 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 00:53:30.263510 15817 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:30.261620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40832 (local address 127.15.147.189:34599)
0806 00:53:30.261750 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.261769 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.262428 (+ 659us) server_negotiation.cc:407] Connection header received
0806 00:53:30.262648 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.262653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.262706 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.262761 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.263364 (+ 603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40832: BlockingRecv error: recv got EOF from 127.0.0.1:40832 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 00:53:30.284161 15817 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:30.282143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40840 (local address 127.15.147.189:34599)
0806 00:53:30.282290 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.282295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.282937 (+ 642us) server_negotiation.cc:407] Connection header received
0806 00:53:30.283230 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.283234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.283293 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.283351 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.284037 (+ 686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40840: BlockingRecv error: recv got EOF from 127.0.0.1:40840 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 00:53:30.320269 15817 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:30.318871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40846 (local address 127.15.147.189:34599)
0806 00:53:30.319012 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.319018 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.319263 (+ 245us) server_negotiation.cc:407] Connection header received
0806 00:53:30.319454 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.319458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.319520 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.319615 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.320137 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40846: BlockingRecv error: recv got EOF from 127.0.0.1:40846 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 00:53:30.388101 15817 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:30.386881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40848 (local address 127.15.147.189:34599)
0806 00:53:30.387020 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.387024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.387211 (+ 187us) server_negotiation.cc:407] Connection header received
0806 00:53:30.387323 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.387326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.387374 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.387457 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.388000 (+ 543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40848: BlockingRecv error: recv got EOF from 127.0.0.1:40848 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 00:53:30.519457 15817 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:30.518340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.15.147.189:34599)
0806 00:53:30.518550 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.518553 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.518574 (+ 21us) server_negotiation.cc:407] Connection header received
0806 00:53:30.518734 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.518737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.518787 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.518863 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.519326 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40854: BlockingRecv error: recv got EOF from 127.0.0.1:40854 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
W20260806 00:53:30.778909 15817 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:30.777685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40864 (local address 127.15.147.189:34599)
0806 00:53:30.777857 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:30.777862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:30.777978 (+ 116us) server_negotiation.cc:407] Connection header received
0806 00:53:30.778162 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:30.778165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:30.778213 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:30.778306 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:30.778782 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40864: BlockingRecv error: recv got EOF from 127.0.0.1:40864 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:53:31.093686 15957 catalog_manager.cc:1259] Loaded cluster ID: ddc81c5a70b94a49b5ae6c0dabb98d2d
I20260806 00:53:31.093778 15957 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7: loading cluster ID for follower catalog manager: success
I20260806 00:53:31.095777 15957 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7: acquiring CA information for follower catalog manager: success
I20260806 00:53:31.097100 15957 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e34b3f6de27a475f9f99daee81029fd7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:53:31.108373 15984 catalog_manager.cc:1259] Loaded cluster ID: ddc81c5a70b94a49b5ae6c0dabb98d2d
I20260806 00:53:31.108475 15984 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9: loading cluster ID for follower catalog manager: success
I20260806 00:53:31.110348 15984 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9: acquiring CA information for follower catalog manager: success
I20260806 00:53:31.111476 15984 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5f203d0b59e1420989f18e6d93c452c9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:53:31.297140 16115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:31.293089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40880 (local address 127.15.147.189:34599)
0806 00:53:31.293319 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:31.293326 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:31.293366 (+ 40us) server_negotiation.cc:407] Connection header received
0806 00:53:31.293505 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:31.293507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:31.293656 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:31.293759 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:31.294022 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:31.294551 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:31.295564 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:31.295845 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:31.295932 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:31.296259 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:31.296801 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:31.296897 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:31.296966 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 00:53:31.297017 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":88,"threads_started":1}
I20260806 00:53:31.304276 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:31.300410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33396 (local address 127.15.147.188:40733)
0806 00:53:31.300677 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:31.300683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:31.300700 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:53:31.300803 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:31.300806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:31.300929 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:31.301058 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:31.301327 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:31.301892 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:31.302777 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:31.303034 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:31.303160 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:31.303473 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:31.303831 (+ 358us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:31.303967 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:31.304013 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 00:53:31.304070 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":78,"threads_started":1}
I20260806 00:53:31.311935 16131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:31.307367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45352 (local address 127.15.147.190:44295)
0806 00:53:31.307752 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:31.307757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:31.307986 (+ 229us) server_negotiation.cc:407] Connection header received
0806 00:53:31.308213 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:31.308217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:31.308416 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:31.308510 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:31.308917 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:31.309714 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:31.310644 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:31.310883 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:31.311003 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:31.311308 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:31.311525 (+ 217us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:31.311623 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:31.311713 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 00:53:31.311765 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":69,"threads_started":1}
I20260806 00:53:31.312533 16115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:31.307407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40890 (local address 127.15.147.189:34599)
0806 00:53:31.307529 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:31.307533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:31.308094 (+ 561us) server_negotiation.cc:407] Connection header received
0806 00:53:31.308179 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:31.308182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:31.308230 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:31.308615 (+ 385us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:31.308716 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:31.309379 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:31.311068 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:31.311449 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:31.311522 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:31.312025 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:31.312119 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:31.312166 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:31.312343 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 00:53:31.312395 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 00:53:31.312564 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:31.307564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33400 (local address 127.15.147.188:40733)
0806 00:53:31.307698 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:31.307702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:31.308055 (+ 353us) server_negotiation.cc:407] Connection header received
0806 00:53:31.308378 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:31.308381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:31.308429 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:31.308510 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:31.308778 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:31.309544 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:31.311068 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:31.311437 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:31.311518 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:31.312055 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:31.312135 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:31.312203 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:31.312387 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 00:53:31.312436 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:53:31.315136 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 15614
I20260806 00:53:31.330265 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 15717
I20260806 00:53:31.341290 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 15819
2026-08-06T00:53:31Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3245 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 00:53:31 dist-test-slave-45dx krb5kdc[16139](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 00:53:31 dist-test-slave-45dx krb5kdc[16139](info): set up 2 sockets
Aug 06 00:53:31 dist-test-slave-45dx krb5kdc[16139](info): commencing operation
krb5kdc: starting...
W20260806 00:53:32.540036 15950 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/test-user.keytab.
Aug 06 00:53:32 dist-test-slave-45dx krb5kdc[16139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T00:53:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T00:53:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.147.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
Entry for principal kudu/127.15.147.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab.
I20260806 00:53:32.665572 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:34027
--webserver_interface=127.15.147.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:34027,127.15.147.189:36055,127.15.147.188:34409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 00:53:32.791548 16159 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 00:53:32.791901 16159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:32.791983 16159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:32.792062 16159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:32.795938 16159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:32.796036 16159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:32.796073 16159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:32.796110 16159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:32.796149 16159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:32.801177 16159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:34027,127.15.147.189:36055,127.15.147.188:34409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.190.keytab
--principal=kudu/127.15.147.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.190:34027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.147.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:32.802285 16159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:32.803355 16159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 00:53:32.812121 16159 server_base.cc:1034] running on GCE node
W20260806 00:53:32.812186 16166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:32.812081 16167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:32.812085 16171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:32.813001 16159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:32.814002 16159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:32.815186 16159 hybrid_clock.cc:648] HybridClock initialized: now 1785977612815178 us; error 36 us; skew 500 ppm
Aug 06 00:53:32 dist-test-slave-45dx krb5kdc[16139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977612, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:32.817893 16159 init.cc:377] Logged in from keytab as kudu/127.15.147.190@KRBTEST.COM (short username kudu)
I20260806 00:53:32.818709 16159 webserver.cc:460] Webserver started at http://127.15.147.190:40399/ using document root <none> and password file <none>
I20260806 00:53:32.819202 16159 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:32.819259 16159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:32.819422 16159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:32.821395 16159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-0/data/instance:
uuid: "536407dcf8724ecab54eda3cd94a6060"
format_stamp: "Formatted at 2026-08-06 00:53:32 on dist-test-slave-45dx"
server_key: "38d08ce61bf8c39891b0ca9edd9534fd"
server_key_iv: "6209f348f35108baba85c10a8f032915"
server_key_version: "encryptionkey@0"
I20260806 00:53:32.822213 16159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-0/wal/instance:
uuid: "536407dcf8724ecab54eda3cd94a6060"
format_stamp: "Formatted at 2026-08-06 00:53:32 on dist-test-slave-45dx"
server_key: "38d08ce61bf8c39891b0ca9edd9534fd"
server_key_iv: "6209f348f35108baba85c10a8f032915"
server_key_version: "encryptionkey@0"
I20260806 00:53:32.825785 16159 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 00:53:32.827948 16180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:32.829152 16159 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:53:32.829279 16159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-0/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-0/wal
uuid: "536407dcf8724ecab54eda3cd94a6060"
format_stamp: "Formatted at 2026-08-06 00:53:32 on dist-test-slave-45dx"
server_key: "38d08ce61bf8c39891b0ca9edd9534fd"
server_key_iv: "6209f348f35108baba85c10a8f032915"
server_key_version: "encryptionkey@0"
I20260806 00:53:32.829384 16159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:32.851882 16159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:32.854928 16159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:32.855082 16159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:32.862221 16159 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.190:34027
I20260806 00:53:32.862530 16252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.190:34027 every 8 connection(s)
I20260806 00:53:32.863266 16159 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-0/data/info.pb
I20260806 00:53:32.866125 16259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:32.869832 16259 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:53:32.873140 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 16159
I20260806 00:53:32.873257 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-0/wal/instance
I20260806 00:53:32.873574 15950 external_mini_cluster.cc:1428] Setting key 12faa6cc31d2e9b2bb9ae0b4f7bf1ed7
Aug 06 00:53:32 dist-test-slave-45dx krb5kdc[16139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977612, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.147.189@KRBTEST.COM; defaulting to no policy
I20260806 00:53:32.885537 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:32.872647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48258 (local address 127.15.147.190:34027)
0806 00:53:32.872932 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:32.872942 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:32.872976 (+ 34us) server_negotiation.cc:407] Connection header received
0806 00:53:32.873747 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:32.873751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:32.873934 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:32.874080 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:32.875174 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:32.876341 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:32.877244 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:32.877859 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:32.878047 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:32.880512 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:32.880522 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:32.880537 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:32.880566 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:32.882547 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:32.883008 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:32.883014 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:32.883016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:32.883158 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:32.883461 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:32.883466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:32.883469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:32.884018 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:32.884150 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:32.884563 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 00:53:32.884790 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":178,"spinlock_wait_cycles":3840,"thread_start_us":96,"threads_started":1}
I20260806 00:53:32.885540 16261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:32.871847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:34027 (local address 127.0.0.1:48258)
0806 00:53:32.872378 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:32.872421 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:32.873400 (+ 979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:32.874145 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:32.874153 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:32.874644 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:32.875018 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:32.875039 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:32.876475 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:32.876478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:32.877129 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:32.877137 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:32.878006 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:32.878009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:32.878097 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:32.878759 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:32.878775 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:32.880343 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:32.882657 (+ 2314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:32.882665 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:32.882673 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:32.882895 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:32.883254 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:32.883256 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:32.883257 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:32.883364 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:32.884128 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:32.884132 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:32.884250 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 00:53:32.884457 (+ 207us) client_negotiation.cc:240] Negotiation successful
0806 00:53:32.884729 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":125,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.189@KRBTEST.COM" created.
I20260806 00:53:32.888106 16259 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:53:32.889014 16261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:32.888678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:36055 (local address 127.0.0.1:53032)
0806 00:53:32.888828 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:32.888920 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.189:36055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 00:53:32.889310 16185 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.147.189:36055: connect: Connection refused (error 111)
I20260806 00:53:32.890316 16261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:32.890129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:36055 (local address 127.0.0.1:53038)
0806 00:53:32.890223 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:32.890266 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.189:36055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 00:53:32.890483 16259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.189:36055: Network error: Client connection negotiation failed: client connection to 127.15.147.189:36055: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.189.keytab.
Entry for principal kudu/127.15.147.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.189.keytab.
I20260806 00:53:32.903609 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.189:36055
--webserver_interface=127.15.147.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.189.keytab
--principal=kudu/127.15.147.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:34027,127.15.147.189:36055,127.15.147.188:34409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:53:32.939738 16259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } attempt: 1
I20260806 00:53:32.940742 16261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:32.940395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:36055 (local address 127.0.0.1:53042)
0806 00:53:32.940594 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:32.940660 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.189:36055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 00:53:32.941459 16261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:32.941247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:36055 (local address 127.0.0.1:53052)
0806 00:53:32.941352 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:32.941402 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.189:36055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 00:53:32.941598 16259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.189:36055: Network error: Client connection negotiation failed: client connection to 127.15.147.189:36055: connect: Connection refused (error 111)
I20260806 00:53:33.009779 16259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } attempt: 2
I20260806 00:53:33.010535 16261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.010245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:36055 (local address 127.0.0.1:53056)
0806 00:53:33.010384 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.010452 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.189:36055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 00:53:33.011266 16261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.011052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:36055 (local address 127.0.0.1:53064)
0806 00:53:33.011155 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.011208 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.189:36055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 00:53:33.011420 16259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.189:36055: Network error: Client connection negotiation failed: client connection to 127.15.147.189:36055: connect: Connection refused (error 111)
W20260806 00:53:33.019438 16266 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 00:53:33.019766 16266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:33.019829 16266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:33.019912 16266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:33.024072 16266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:33.024160 16266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:33.024199 16266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:33.024235 16266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:33.024272 16266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:33.029712 16266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:34027,127.15.147.189:36055,127.15.147.188:34409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.189.keytab
--principal=kudu/127.15.147.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.189:36055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.147.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:33.030807 16266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:33.031832 16266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:33.040597 16273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:33.040597 16275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:33.040644 16272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:33.040802 16266 server_base.cc:1034] running on GCE node
I20260806 00:53:33.041466 16266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:33.042526 16266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:33.043716 16266 hybrid_clock.cc:648] HybridClock initialized: now 1785977613043688 us; error 40 us; skew 500 ppm
Aug 06 00:53:33 dist-test-slave-45dx krb5kdc[16139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977613, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:33.050107 16266 init.cc:377] Logged in from keytab as kudu/127.15.147.189@KRBTEST.COM (short username kudu)
I20260806 00:53:33.051025 16266 webserver.cc:460] Webserver started at http://127.15.147.189:46369/ using document root <none> and password file <none>
I20260806 00:53:33.051579 16266 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:33.051671 16266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:33.051916 16266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:33.053985 16266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-1/data/instance:
uuid: "2955dd58f19745e69d87b228f8bf59ab"
format_stamp: "Formatted at 2026-08-06 00:53:33 on dist-test-slave-45dx"
server_key: "48930282015ab01b1c1011937b9d92f5"
server_key_iv: "39f965bfaa9a762b4849cd2bfcf40963"
server_key_version: "encryptionkey@0"
I20260806 00:53:33.054534 16266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-1/wal/instance:
uuid: "2955dd58f19745e69d87b228f8bf59ab"
format_stamp: "Formatted at 2026-08-06 00:53:33 on dist-test-slave-45dx"
server_key: "48930282015ab01b1c1011937b9d92f5"
server_key_iv: "39f965bfaa9a762b4849cd2bfcf40963"
server_key_version: "encryptionkey@0"
I20260806 00:53:33.058159 16266 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 00:53:33.060577 16283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:33.061728 16266 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 00:53:33.061909 16266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-1/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-1/wal
uuid: "2955dd58f19745e69d87b228f8bf59ab"
format_stamp: "Formatted at 2026-08-06 00:53:33 on dist-test-slave-45dx"
server_key: "48930282015ab01b1c1011937b9d92f5"
server_key_iv: "39f965bfaa9a762b4849cd2bfcf40963"
server_key_version: "encryptionkey@0"
I20260806 00:53:33.062023 16266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:33.078625 16266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:33.081525 16266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:33.081724 16266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:33.088814 16266 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.189:36055
I20260806 00:53:33.089104 16363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.189:36055 every 8 connection(s)
I20260806 00:53:33.089775 16266 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-1/data/info.pb
I20260806 00:53:33.090607 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 16266
I20260806 00:53:33.090725 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-1/wal/instance
I20260806 00:53:33.091038 15950 external_mini_cluster.cc:1428] Setting key 62b928a82b709a31363a3bb951b7b8df
I20260806 00:53:33.093099 16364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:33.097014 16364 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.147.188@KRBTEST.COM; defaulting to no policy
I20260806 00:53:33.102576 16259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.147.188@KRBTEST.COM" created.
Aug 06 00:53:33 dist-test-slave-45dx krb5kdc[16139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977613, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.189@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
Aug 06 00:53:33 dist-test-slave-45dx krb5kdc[16139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977612, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for kudu/127.15.147.189@KRBTEST.COM
I20260806 00:53:33.113520 16261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.103061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:36055 (local address 127.0.0.1:53068)
0806 00:53:33.103246 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.103262 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:33.103333 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:33.103981 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:33.103984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:33.104180 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:33.104277 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:33.104283 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.105451 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.105456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:33.106195 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:33.106205 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.107007 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.107014 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:33.107151 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.107733 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:33.107753 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:33.109286 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:33.111275 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:33.111279 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:33.111281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:33.111483 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:33.111948 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:33.111952 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:33.111955 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:33.112062 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:33.112862 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:33.112868 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:33.112989 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 00:53:33.113142 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 00:53:33.113274 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260806 00:53:33.114434 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.098937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48270 (local address 127.15.147.190:34027)
0806 00:53:33.099079 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.099083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.100126 (+ 1043us) server_negotiation.cc:407] Connection header received
0806 00:53:33.101120 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.101124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.101191 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.101288 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.102443 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.103200 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.105005 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.105392 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.105483 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.108466 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:33.108491 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:33.108497 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:33.108531 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:33.111148 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.112059 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.112061 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.112063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.112163 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.112539 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.112543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.112544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.112741 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:33.112842 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:33.114176 (+ 1334us) server_negotiation.cc:299] Negotiation successful
0806 00:53:33.114287 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:53:33.114567 16369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.099440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:34027 (local address 127.0.0.1:48270)
0806 00:53:33.099976 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.100016 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:33.100909 (+ 893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:33.101495 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:33.101502 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:33.101838 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:33.102295 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:33.102308 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.103348 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.103353 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:33.104867 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:33.104882 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.105641 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.105645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:33.105848 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.106539 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:33.106558 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:33.108284 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:33.111604 (+ 3320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:33.111610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:33.111616 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:33.111950 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:33.112253 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:33.112256 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:33.112258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:33.112410 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:33.113752 (+ 1342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:33.113760 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:33.113896 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 00:53:33.114042 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 00:53:33.114184 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"spinlock_wait_cycles":1792,"thread_start_us":163,"threads_started":1}
I20260806 00:53:33.114574 16370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.103153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53068 (local address 127.15.147.189:36055)
0806 00:53:33.103599 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.103608 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.103630 (+ 22us) server_negotiation.cc:407] Connection header received
0806 00:53:33.103664 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.103668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.103820 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.103991 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.104421 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.105305 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.106366 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.106861 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.106961 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.109436 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:33.109443 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:33.109457 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:33.109485 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:33.111155 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.111600 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.111606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.111612 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.111826 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.112170 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.112173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.112175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.112730 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:33.112873 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:33.113269 (+ 396us) server_negotiation.cc:299] Negotiation successful
0806 00:53:33.113486 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"spinlock_wait_cycles":3456,"thread_start_us":101,"threads_started":1}
I20260806 00:53:33.116627 16364 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:53:33.117389 16259 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:53:33.118129 16261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.117815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34409 (local address 127.0.0.1:39950)
0806 00:53:33.118011 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.118066 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260806 00:53:33.118871 16261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.118626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34409 (local address 127.0.0.1:39960)
0806 00:53:33.118772 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.118820 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260806 00:53:33.119019 16259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.188:34409: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.147.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.188.keytab.
Entry for principal kudu/127.15.147.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.188.keytab.
Aug 06 00:53:33 dist-test-slave-45dx krb5kdc[16139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977613, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.189@KRBTEST.COM for kudu/127.15.147.189@KRBTEST.COM
I20260806 00:53:33.121357 15950 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3y4EW9/build/debug/bin/kudu
/tmp/dist-test-task3y4EW9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.188:34409
--webserver_interface=127.15.147.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.188.keytab
--principal=kudu/127.15.147.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.147.148:34993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.147.190:34027,127.15.147.189:36055,127.15.147.188:34409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 00:53:33.125339 16370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.117253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53076 (local address 127.15.147.189:36055)
0806 00:53:33.117384 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.117389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.117404 (+ 15us) server_negotiation.cc:407] Connection header received
0806 00:53:33.117513 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.117516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.117559 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.117635 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.118019 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.118675 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.119565 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.119797 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.119907 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.121875 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:33.121878 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:33.121880 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:33.121907 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:33.123547 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.123952 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.123957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.123960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.124059 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.124336 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.124339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.124341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.124586 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:33.124664 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:33.125006 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 00:53:33.125210 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":103,"server-negotiator.queue_time_us":63}
I20260806 00:53:33.125432 16369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.117191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:36055 (local address 127.0.0.1:53076)
0806 00:53:33.117324 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.117339 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:33.117419 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:33.117644 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:33.117646 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:33.117772 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:33.117911 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:33.117916 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.118803 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.118807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:33.119469 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:33.119478 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.119929 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.119933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:33.120022 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.120483 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:33.120518 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:33.121733 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:33.123661 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:33.123663 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:33.123665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:33.123847 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:33.124152 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:33.124154 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:33.124155 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:33.124228 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:33.124689 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:33.124695 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:33.124817 (+ 122us) client_negotiation.cc:769] Sending connection context
0806 00:53:33.124949 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 00:53:33.125128 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 00:53:33.126291 16364 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:53:33.127065 16369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.126733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34409 (local address 127.0.0.1:39974)
0806 00:53:33.126885 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.126981 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 00:53:33.127696 16289 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
I20260806 00:53:33.128721 16369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.128441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34409 (local address 127.0.0.1:39984)
0806 00:53:33.128576 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.128641 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 00:53:33.128883 16364 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.188:34409: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
I20260806 00:53:33.150187 16259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } attempt: 1
I20260806 00:53:33.150960 16261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.150700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34409 (local address 127.0.0.1:39996)
0806 00:53:33.150834 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.150888 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 00:53:33.151675 16261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.151494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34409 (local address 127.0.0.1:40002)
0806 00:53:33.151585 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.151626 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 00:53:33.151865 16259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.188:34409: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
I20260806 00:53:33.178176 16364 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } attempt: 1
I20260806 00:53:33.179144 16369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.178778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34409 (local address 127.0.0.1:40016)
0806 00:53:33.178967 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.179044 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 00:53:33.179924 16369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.179689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34409 (local address 127.0.0.1:40024)
0806 00:53:33.179804 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.179864 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 00:53:33.180086 16364 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.188:34409: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
I20260806 00:53:33.227104 16259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } attempt: 2
I20260806 00:53:33.228057 16261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.227696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34409 (local address 127.0.0.1:40030)
0806 00:53:33.227900 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.227965 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 00:53:33.228842 16261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.228623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34409 (local address 127.0.0.1:40038)
0806 00:53:33.228735 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.228787 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 00:53:33.228984 16259 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.188:34409: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
W20260806 00:53:33.238348 16372 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 00:53:33.238687 16372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 00:53:33.238750 16372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 00:53:33.238827 16372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 00:53:33.242830 16372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 00:53:33.242909 16372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 00:53:33.242967 16372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 00:53:33.243023 16372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 00:53:33.243057 16372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 00:53:33.248292 16364 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } attempt: 2
I20260806 00:53:33.248413 16372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.147.148:34993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.147.190:34027,127.15.147.189:36055,127.15.147.188:34409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/krb5kdc/kudu_127.15.147.188.keytab
--principal=kudu/127.15.147.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.147.188:34409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.147.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 00:07:19 UTC on ccycloud.quasar-mgiraf.root.comops.site
build id 4569411
I20260806 00:53:33.249197 16369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.248858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34409 (local address 127.0.0.1:40044)
0806 00:53:33.249034 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.249107 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 00:53:33.249812 16372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 00:53:33.249961 16369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.249706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34409 (local address 127.0.0.1:40052)
0806 00:53:33.249818 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.249895 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 00:53:33.250092 16364 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.147.188:34409: Network error: Client connection negotiation failed: client connection to 127.15.147.188:34409: connect: Connection refused (error 111)
I20260806 00:53:33.250905 16372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 00:53:33.259776 16380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 00:53:33.259802 16379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:33.259933 16372 server_base.cc:1034] running on GCE node
W20260806 00:53:33.259799 16382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 00:53:33.260725 16372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 00:53:33.261812 16372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 00:53:33.263011 16372 hybrid_clock.cc:648] HybridClock initialized: now 1785977613262989 us; error 43 us; skew 500 ppm
Aug 06 00:53:33 dist-test-slave-45dx krb5kdc[16139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977613, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 00:53:33.265826 16372 init.cc:377] Logged in from keytab as kudu/127.15.147.188@KRBTEST.COM (short username kudu)
I20260806 00:53:33.266639 16372 webserver.cc:460] Webserver started at http://127.15.147.188:39137/ using document root <none> and password file <none>
I20260806 00:53:33.267191 16372 fs_manager.cc:359] Metadata directory not provided
I20260806 00:53:33.267251 16372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 00:53:33.267407 16372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 00:53:33.269477 16372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-2/data/instance:
uuid: "66ffc2f6581744b19b2375a500195e8d"
format_stamp: "Formatted at 2026-08-06 00:53:33 on dist-test-slave-45dx"
server_key: "b8112d1297a32c180e1b6b519476c27a"
server_key_iv: "7ffdd88688d7de5d3cdba29d81ec9b89"
server_key_version: "encryptionkey@0"
I20260806 00:53:33.269979 16372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-2/wal/instance:
uuid: "66ffc2f6581744b19b2375a500195e8d"
format_stamp: "Formatted at 2026-08-06 00:53:33 on dist-test-slave-45dx"
server_key: "b8112d1297a32c180e1b6b519476c27a"
server_key_iv: "7ffdd88688d7de5d3cdba29d81ec9b89"
server_key_version: "encryptionkey@0"
I20260806 00:53:33.273319 16372 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 00:53:33.275586 16389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 00:53:33.276791 16372 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 00:53:33.276940 16372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-2/data,/tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-2/wal
uuid: "66ffc2f6581744b19b2375a500195e8d"
format_stamp: "Formatted at 2026-08-06 00:53:33 on dist-test-slave-45dx"
server_key: "b8112d1297a32c180e1b6b519476c27a"
server_key_iv: "7ffdd88688d7de5d3cdba29d81ec9b89"
server_key_version: "encryptionkey@0"
I20260806 00:53:33.277052 16372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 00:53:33.291247 16372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 00:53:33.293910 16372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 00:53:33.294128 16372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 00:53:33.301093 16372 rpc_server.cc:307] RPC server started. Bound to: 127.15.147.188:34409
I20260806 00:53:33.301403 16471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.147.188:34409 every 8 connection(s)
I20260806 00:53:33.302047 16372 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-2/data/info.pb
I20260806 00:53:33.304811 16472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 00:53:33.308575 16472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:53:33.309211 15950 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3y4EW9/build/debug/bin/kudu as pid 16372
I20260806 00:53:33.309360 15950 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3y4EW9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785977317010501-15950-0/minicluster-data/master-2/wal/instance
I20260806 00:53:33.309633 15950 external_mini_cluster.cc:1428] Setting key 923b0738bd8906322431417bbe5ce850
Aug 06 00:53:33 dist-test-slave-45dx krb5kdc[16139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977613, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.188@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
Aug 06 00:53:33 dist-test-slave-45dx krb5kdc[16139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.188@KRBTEST.COM
I20260806 00:53:33.322870 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.310512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48286 (local address 127.15.147.190:34027)
0806 00:53:33.310644 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.310649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.311481 (+ 832us) server_negotiation.cc:407] Connection header received
0806 00:53:33.312277 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.312280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.312364 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.312455 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.313539 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.314317 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.315695 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.315915 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.316036 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.318809 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:33.318812 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:33.318814 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:33.318839 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:33.320614 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.321191 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.321197 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.321201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.321290 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.321665 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.321668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.321669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.321937 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:33.322055 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:33.322558 (+ 503us) server_negotiation.cc:299] Negotiation successful
0806 00:53:33.322705 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 00:53:33.323683 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.310869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.190:34027 (local address 127.0.0.1:48286)
0806 00:53:33.311323 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.311370 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:33.312066 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:33.312535 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:33.312540 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:33.313013 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:33.313383 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:33.313403 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.314435 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.314437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:33.315555 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:33.315566 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.316074 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.316078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:33.316237 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.317045 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:33.317061 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:33.318641 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:33.320719 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:33.320727 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:33.320734 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:33.321091 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:33.321389 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:33.321392 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:33.321393 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:33.321567 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:33.322063 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:33.322068 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:33.322287 (+ 219us) client_negotiation.cc:769] Sending connection context
0806 00:53:33.322469 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 00:53:33.322789 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":3,"thread_start_us":123,"threads_started":1}
I20260806 00:53:33.323683 16476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.310870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40066 (local address 127.15.147.188:34409)
0806 00:53:33.311304 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.311318 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.311350 (+ 32us) server_negotiation.cc:407] Connection header received
0806 00:53:33.312048 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.312063 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.312300 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.312500 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.313418 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.314617 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.315711 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.316093 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.316242 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.319288 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:33.319293 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:33.319319 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:33.319348 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:33.321158 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.321642 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.321648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.321651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.321739 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.322040 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.322043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.322045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.322292 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:33.322455 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:33.322732 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 00:53:33.322861 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":309,"thread_start_us":159,"threads_started":1}
I20260806 00:53:33.324900 16472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:53:33.328151 16259 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } attempt: 3
Aug 06 00:53:33 dist-test-slave-45dx krb5kdc[16139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977613, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.188@KRBTEST.COM for kudu/127.15.147.189@KRBTEST.COM
Aug 06 00:53:33 dist-test-slave-45dx krb5kdc[16139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.190@KRBTEST.COM
Aug 06 00:53:33 dist-test-slave-45dx krb5kdc[16139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977612, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.190@KRBTEST.COM for kudu/127.15.147.188@KRBTEST.COM
I20260806 00:53:33.335464 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.326107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48290 (local address 127.15.147.190:34027)
0806 00:53:33.326217 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.326221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.326282 (+ 61us) server_negotiation.cc:407] Connection header received
0806 00:53:33.326404 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.326427 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.326475 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.326554 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.327080 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.327665 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.328666 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.328978 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.329076 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.331747 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:33.331753 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:33.331757 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:33.331781 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:33.333418 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.333904 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.333908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.333912 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.334008 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.334499 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.334505 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.334508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.334747 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:33.335013 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:33.335155 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 00:53:33.335276 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 00:53:33.335767 16370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.325540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53080 (local address 127.15.147.189:36055)
0806 00:53:33.325690 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.325695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.325707 (+ 12us) server_negotiation.cc:407] Connection header received
0806 00:53:33.325752 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.325754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.325810 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.325897 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.326483 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.327077 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.327865 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.328097 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.328175 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.331959 (+ 3784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:33.331963 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:33.331966 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:33.332001 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:33.333832 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.334365 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.334369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.334371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.334466 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.334822 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.334828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.334830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.335063 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:33.335156 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:33.335493 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 00:53:33.335626 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 00:53:33.336266 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.325399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.189:36055 (local address 127.0.0.1:53080)
0806 00:53:33.325501 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.325516 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:33.325603 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:33.325910 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:33.325913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:33.326089 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:33.326348 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:33.326358 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.327194 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.327197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:33.327741 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:33.327750 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.328875 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.328879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:33.328990 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.329516 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:33.329530 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:33.331537 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:33.333953 (+ 2416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:33.333956 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:33.333958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:33.334229 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:33.334585 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:33.334587 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:33.334589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:33.334701 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:33.335201 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:33.335204 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:33.335298 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 00:53:33.335961 (+ 663us) client_negotiation.cc:240] Negotiation successful
0806 00:53:33.336105 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 00:53:33.337237 16261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.328519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34409 (local address 127.0.0.1:40068)
0806 00:53:33.328681 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.328695 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:33.328765 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:33.328983 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:33.328986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:33.329173 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:33.329277 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:33.329285 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.330281 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.330285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:33.330821 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:33.330833 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.331337 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.331341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:33.331420 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.331857 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:33.331874 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:33.333523 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:33.335616 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:33.335621 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:33.335623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:33.335906 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:33.336219 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:33.336222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:33.336224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:33.336344 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:33.336777 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:33.336780 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:33.336875 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 00:53:33.336958 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 00:53:33.337089 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260806 00:53:33.337234 16472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } has no permanent_uuid. Determining permanent_uuid...
I20260806 00:53:33.337749 16476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.328599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40068 (local address 127.15.147.188:34409)
0806 00:53:33.328698 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.328702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.328733 (+ 31us) server_negotiation.cc:407] Connection header received
0806 00:53:33.328838 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.328840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.328875 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.328964 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.329424 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.330170 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.330935 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.331184 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.331334 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.333625 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:33.333629 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:33.333631 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:33.333699 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:33.335488 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.336030 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.336034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.336035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.336119 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.336452 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.336455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.336456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.336658 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:33.337409 (+ 751us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:33.337480 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 00:53:33.337596 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 00:53:33.341259 16364 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } attempt: 3
I20260806 00:53:33.341352 16259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060: Bootstrap starting.
Aug 06 00:53:33 dist-test-slave-45dx krb5kdc[16139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.147.189@KRBTEST.COM
Aug 06 00:53:33 dist-test-slave-45dx krb5kdc[16139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977613, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.188@KRBTEST.COM for kudu/127.15.147.188@KRBTEST.COM
I20260806 00:53:33.344599 16259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:33.345642 16259 log.cc:824] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060: Log is configured to *not* fsync() on all Append() calls
Aug 06 00:53:33 dist-test-slave-45dx krb5kdc[16139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785977613, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.147.189@KRBTEST.COM for kudu/127.15.147.188@KRBTEST.COM
I20260806 00:53:33.347971 16370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.336772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53082 (local address 127.15.147.189:36055)
0806 00:53:33.336886 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.336890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.336901 (+ 11us) server_negotiation.cc:407] Connection header received
0806 00:53:33.336937 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.336943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.336984 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.337055 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.337809 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.338477 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.339555 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.339953 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.340080 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.343714 (+ 3634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:33.343718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:33.343720 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:33.343745 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:33.345655 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.346183 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.346187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.346188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.346283 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.346629 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.346636 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.346641 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.346842 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:33.347058 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:33.347709 (+ 651us) server_negotiation.cc:299] Negotiation successful
0806 00:53:33.347822 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 00:53:33.348042 16259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060: No bootstrap required, opened a new log
I20260806 00:53:33.348531 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.337643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34409 (local address 127.0.0.1:40076)
0806 00:53:33.337758 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.337774 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:33.337936 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:33.338222 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:33.338225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:33.338376 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:33.338450 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:33.338455 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.339417 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.339420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:33.340081 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:33.340092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.341436 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.341440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:33.341549 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.341981 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:33.342000 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:33.344642 (+ 2642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:33.346844 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:33.346846 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:33.346848 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:33.347072 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:33.347341 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:33.347346 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:33.347347 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:33.347430 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:33.347992 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:33.347995 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:33.348102 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 00:53:33.348199 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 00:53:33.348370 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 00:53:33.349020 16476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.337826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40076 (local address 127.15.147.188:34409)
0806 00:53:33.338017 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.338020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.338034 (+ 14us) server_negotiation.cc:407] Connection header received
0806 00:53:33.338071 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.338073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.338113 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.338194 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.338600 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.339308 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.340199 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.341204 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.341331 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.344764 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:33.344768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:33.344770 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:33.344792 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:33.346736 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.347163 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.347165 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.347167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.347245 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.347544 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.347547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.347552 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.347857 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:33.347957 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:33.348730 (+ 773us) server_negotiation.cc:299] Negotiation successful
0806 00:53:33.348859 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":121,"threads_started":1}
I20260806 00:53:33.351442 15950 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 00:53:33.352169 16369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.341615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.147.188:34409 (local address 127.0.0.1:40086)
0806 00:53:33.341796 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 00:53:33.341807 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 00:53:33.341892 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 00:53:33.342502 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 00:53:33.342506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 00:53:33.342711 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 00:53:33.342795 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:33.342800 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.343633 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.343637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:33.344138 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 00:53:33.344145 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.344769 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.344773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 00:53:33.344866 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.345404 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 00:53:33.345422 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 00:53:33.347031 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 00:53:33.350492 (+ 3461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:33.350495 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:33.350497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:33.350694 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:33.350996 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 00:53:33.350999 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 00:53:33.351001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 00:53:33.351105 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 00:53:33.351673 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 00:53:33.351676 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 00:53:33.351776 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 00:53:33.351892 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 00:53:33.352023 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260806 00:53:33.352448 16478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.342134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40086 (local address 127.15.147.188:34409)
0806 00:53:33.342251 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.342262 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.342273 (+ 11us) server_negotiation.cc:407] Connection header received
0806 00:53:33.342306 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.342308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.342347 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.342885 (+ 538us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.342959 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.343520 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.344258 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.344635 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.345067 (+ 432us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.347176 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:33.347179 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:33.347181 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:33.347206 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:33.350370 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.350796 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.350800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.350802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.350891 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.351335 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.351338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.351339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.351564 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:33.351657 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:33.351980 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 00:53:33.352105 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 00:53:33.352501 16259 raft_consensus.cc:348] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "536407dcf8724ecab54eda3cd94a6060" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } } peers { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } } peers { permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } }
I20260806 00:53:33.352689 16217 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "536407dcf8724ecab54eda3cd94a6060"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48290
I20260806 00:53:33.352778 16259 raft_consensus.cc:729] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 536407dcf8724ecab54eda3cd94a6060, State: Initialized, Role: FOLLOWER
I20260806 00:53:33.355544 16472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d: Bootstrap starting.
I20260806 00:53:33.355544 16364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab: Bootstrap starting.
I20260806 00:53:33.353129 16259 consensus_queue.cc:260] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "536407dcf8724ecab54eda3cd94a6060" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } } peers { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } } peers { permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } }
I20260806 00:53:33.356997 16480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "536407dcf8724ecab54eda3cd94a6060" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } } peers { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } } peers { permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } } }
I20260806 00:53:33.357282 16480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:53:33.357815 16259 sys_catalog.cc:564] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:33.357908 16217 raft_consensus.cc:482] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 00:53:33.358011 16217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:33.358275 16364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:33.358275 16472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d: Neither blocks nor log segments found. Creating new log.
I20260806 00:53:33.359153 16364 log.cc:824] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:33.359153 16472 log.cc:824] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d: Log is configured to *not* fsync() on all Append() calls
I20260806 00:53:33.359395 16217 raft_consensus.cc:504] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "536407dcf8724ecab54eda3cd94a6060" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } } peers { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } } peers { permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } }
I20260806 00:53:33.361807 16364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab: No bootstrap required, opened a new log
I20260806 00:53:33.365705 16472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d: No bootstrap required, opened a new log
I20260806 00:53:33.369686 16472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "536407dcf8724ecab54eda3cd94a6060" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } } peers { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } } peers { permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } }
I20260806 00:53:33.369949 16472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ffc2f6581744b19b2375a500195e8d, State: Initialized, Role: FOLLOWER
I20260806 00:53:33.370277 16472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "536407dcf8724ecab54eda3cd94a6060" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } } peers { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } } peers { permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } }
I20260806 00:53:33.370785 16503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "536407dcf8724ecab54eda3cd94a6060" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } } peers { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } } peers { permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } } }
I20260806 00:53:33.370950 16503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:53:33.371646 16472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:33.371706 16433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "536407dcf8724ecab54eda3cd94a6060" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "66ffc2f6581744b19b2375a500195e8d"
I20260806 00:53:33.371915 16433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:33.360723 16217 leader_election.cc:288] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [CANDIDATE]: Term 1 election: Requested vote from peers 2955dd58f19745e69d87b228f8bf59ab (127.15.147.189:36055), 66ffc2f6581744b19b2375a500195e8d (127.15.147.188:34409)
I20260806 00:53:33.373903 16433 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 536407dcf8724ecab54eda3cd94a6060 in term 1.
I20260806 00:53:33.374070 16364 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "536407dcf8724ecab54eda3cd94a6060" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } } peers { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } } peers { permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } }
I20260806 00:53:33.374302 16364 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2955dd58f19745e69d87b228f8bf59ab, State: Initialized, Role: FOLLOWER
I20260806 00:53:33.374326 16183 leader_election.cc:302] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 536407dcf8724ecab54eda3cd94a6060, 66ffc2f6581744b19b2375a500195e8d; no voters:
I20260806 00:53:33.374605 16364 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "536407dcf8724ecab54eda3cd94a6060" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } } peers { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } } peers { permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } }
I20260806 00:53:33.375590 16480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 00:53:33.375821 16480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [term 1 LEADER]: Becoming Leader. State: Replica: 536407dcf8724ecab54eda3cd94a6060, State: Running, Role: LEADER
I20260806 00:53:33.375888 16364 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab [sys.catalog]: configured and running, proceeding with master startup.
I20260806 00:53:33.376122 16480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "536407dcf8724ecab54eda3cd94a6060" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } } peers { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } } peers { permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } }
I20260806 00:53:33.376860 16323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "536407dcf8724ecab54eda3cd94a6060" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2955dd58f19745e69d87b228f8bf59ab"
I20260806 00:53:33.377070 16323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 00:53:33.377827 16480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 536407dcf8724ecab54eda3cd94a6060. Latest consensus state: current_term: 1 leader_uuid: "536407dcf8724ecab54eda3cd94a6060" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "536407dcf8724ecab54eda3cd94a6060" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } } peers { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } } peers { permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } } }
I20260806 00:53:33.377921 16480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:33.378422 16323 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 536407dcf8724ecab54eda3cd94a6060 in term 1.
W20260806 00:53:33.363018 16499 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:53:33.378882 16499 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 00:53:33.383816 16546 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:53:33.383899 16546 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:53:33.379634 16159 master_runner.cc:186] Error getting master registration for 127.15.147.189:36055: OK, instance_id { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" instance_seqno: 1785977613086908 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 00:53:33.385046 16159 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 00:53:33.390901 16550 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 00:53:33.390990 16550 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 00:53:33.391278 16531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 00:53:33.391254 16506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "536407dcf8724ecab54eda3cd94a6060" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } } peers { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } } peers { permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } } }
I20260806 00:53:33.391361 16506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:53:33.393479 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.376441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48304 (local address 127.15.147.190:34027)
0806 00:53:33.378165 (+ 1724us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.378171 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.380180 (+ 2009us) server_negotiation.cc:407] Connection header received
0806 00:53:33.384272 (+ 4092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.384275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.384348 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.384400 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.385541 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.386226 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.387354 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.387667 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.387737 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.389669 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:33.389673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:33.389676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:33.389701 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:33.391441 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.391998 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.392002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.392004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.392089 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.392445 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.392449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.392451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.392674 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:33.392796 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:33.393171 (+ 375us) server_negotiation.cc:299] Negotiation successful
0806 00:53:33.393288 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1650}
I20260806 00:53:33.394207 16531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 00:53:33.408233 16478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.380563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40090 (local address 127.15.147.188:34409)
0806 00:53:33.380686 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.380689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.395192 (+ 14503us) server_negotiation.cc:407] Connection header received
0806 00:53:33.395363 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.395366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.395414 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.395463 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.396378 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.397258 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.398369 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.398742 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.398817 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.401690 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:33.401693 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:33.401696 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:33.401724 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:33.403511 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.405975 (+ 2464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.405979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.405981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.406078 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.406775 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.406779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.406781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.407005 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:33.407126 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:33.407926 (+ 800us) server_negotiation.cc:299] Negotiation successful
0806 00:53:33.408059 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 00:53:33.411167 16370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:33.379203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53088 (local address 127.15.147.189:36055)
0806 00:53:33.379306 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.379310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.393375 (+ 14065us) server_negotiation.cc:407] Connection header received
0806 00:53:33.396501 (+ 3126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.396505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.396571 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.396621 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.397526 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.398259 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.399308 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:33.399667 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:33.399750 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:33.404027 (+ 4277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 00:53:33.404030 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 00:53:33.404033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 00:53:33.404062 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 00:53:33.405836 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.407436 (+ 1600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.407439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.407441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.407545 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 00:53:33.410087 (+ 2542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 00:53:33.410092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 00:53:33.410094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 00:53:33.410327 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 00:53:33.410441 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:33.410875 (+ 434us) server_negotiation.cc:299] Negotiation successful
0806 00:53:33.411000 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 00:53:33.416661 16433 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d [term 1 FOLLOWER]: Refusing update from remote peer 536407dcf8724ecab54eda3cd94a6060: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:33.417564 16511 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:33.424196 16503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 536407dcf8724ecab54eda3cd94a6060. Latest consensus state: current_term: 1 leader_uuid: "536407dcf8724ecab54eda3cd94a6060" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "536407dcf8724ecab54eda3cd94a6060" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } } peers { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } } peers { permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } } }
I20260806 00:53:33.424345 16503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:53:33.426613 16323 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab [term 1 FOLLOWER]: Refusing update from remote peer 536407dcf8724ecab54eda3cd94a6060: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 00:53:33.427192 16511 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 00:53:33.430863 16506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 536407dcf8724ecab54eda3cd94a6060. Latest consensus state: current_term: 1 leader_uuid: "536407dcf8724ecab54eda3cd94a6060" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "536407dcf8724ecab54eda3cd94a6060" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } } peers { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } } peers { permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } } }
I20260806 00:53:33.430989 16506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:53:33.445794 16480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "536407dcf8724ecab54eda3cd94a6060" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "536407dcf8724ecab54eda3cd94a6060" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } } peers { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } } peers { permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } } }
I20260806 00:53:33.445937 16480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:33.446388 16503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "536407dcf8724ecab54eda3cd94a6060" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "536407dcf8724ecab54eda3cd94a6060" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } } peers { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } } peers { permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } } }
I20260806 00:53:33.446506 16503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:53:33.446698 16556 mvcc.cc:205] Tried to move back new op lower bound from 7315364304546820096 to 7315364304391913472. Current Snapshot: MvccSnapshot[applied={T|T < 7315364304546820096}]
I20260806 00:53:33.447490 16558 mvcc.cc:205] Tried to move back new op lower bound from 7315364304546820096 to 7315364304391913472. Current Snapshot: MvccSnapshot[applied={T|T < 7315364304546820096}]
I20260806 00:53:33.447448 16511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "536407dcf8724ecab54eda3cd94a6060" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "536407dcf8724ecab54eda3cd94a6060" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } } peers { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } } peers { permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } } }
I20260806 00:53:33.447577 16511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060 [sys.catalog]: This master's current role is: LEADER
I20260806 00:53:33.448680 16506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "536407dcf8724ecab54eda3cd94a6060" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "536407dcf8724ecab54eda3cd94a6060" member_type: VOTER last_known_addr { host: "127.15.147.190" port: 34027 } } peers { permanent_uuid: "2955dd58f19745e69d87b228f8bf59ab" member_type: VOTER last_known_addr { host: "127.15.147.189" port: 36055 } } peers { permanent_uuid: "66ffc2f6581744b19b2375a500195e8d" member_type: VOTER last_known_addr { host: "127.15.147.188" port: 34409 } } }
I20260806 00:53:33.448804 16506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab [sys.catalog]: This master's current role is: FOLLOWER
I20260806 00:53:33.456416 16560 mvcc.cc:205] Tried to move back new op lower bound from 7315364304546820096 to 7315364304391913472. Current Snapshot: MvccSnapshot[applied={T|T < 7315364304546820096}]
I20260806 00:53:33.461751 16531 catalog_manager.cc:1347] Generated new cluster ID: 7e5f2c68708440c283814da901a2368c
I20260806 00:53:33.461845 16531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 00:53:33.499850 16531 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 00:53:33.501108 16531 catalog_manager.cc:1504] Loading token signing keys...
I20260806 00:53:33.515018 16531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 536407dcf8724ecab54eda3cd94a6060: Generated new TSK 0
I20260806 00:53:33.515836 16531 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 00:53:33.554988 16370 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:33.551814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53098 (local address 127.15.147.189:36055)
0806 00:53:33.551945 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.551950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.553828 (+ 1878us) server_negotiation.cc:407] Connection header received
0806 00:53:33.554054 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.554058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.554114 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.554170 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.554846 (+ 676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53098: BlockingRecv error: recv got EOF from 127.0.0.1:53098 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 00:53:33.556470 16262 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:33.552592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48314 (local address 127.15.147.190:34027)
0806 00:53:33.552709 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.552713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.555500 (+ 2787us) server_negotiation.cc:407] Connection header received
0806 00:53:33.555645 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.555648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.555708 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.555801 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.556336 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48314: BlockingRecv error: recv got EOF from 127.0.0.1:48314 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 00:53:33.556947 16478 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:33.553250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40106 (local address 127.15.147.188:34409)
0806 00:53:33.553381 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.553387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.555871 (+ 2484us) server_negotiation.cc:407] Connection header received
0806 00:53:33.556053 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.556057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.556110 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.556165 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.556817 (+ 652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40106: BlockingRecv error: recv got EOF from 127.0.0.1:40106 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 00:53:33.562156 16370 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:33.560236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53110 (local address 127.15.147.189:36055)
0806 00:53:33.560408 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.560412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.561103 (+ 691us) server_negotiation.cc:407] Connection header received
0806 00:53:33.561370 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.561374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.561451 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.561506 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.562055 (+ 549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53110: BlockingRecv error: recv got EOF from 127.0.0.1:53110 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260806 00:53:33.567624 16370 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:33.565942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53118 (local address 127.15.147.189:36055)
0806 00:53:33.566059 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.566063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.566697 (+ 634us) server_negotiation.cc:407] Connection header received
0806 00:53:33.566892 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.566895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.566942 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.566989 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.567526 (+ 537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53118: BlockingRecv error: recv got EOF from 127.0.0.1:53118 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 00:53:33.573185 16370 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:33.571665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53132 (local address 127.15.147.189:36055)
0806 00:53:33.571759 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.571763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.572334 (+ 571us) server_negotiation.cc:407] Connection header received
0806 00:53:33.572518 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.572521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.572567 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.572616 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.573089 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53132: BlockingRecv error: recv got EOF from 127.0.0.1:53132 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 00:53:33.580916 16370 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:33.579093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53134 (local address 127.15.147.189:36055)
0806 00:53:33.579224 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.579228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.579780 (+ 552us) server_negotiation.cc:407] Connection header received
0806 00:53:33.579996 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.579999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.580051 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.580147 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.580802 (+ 655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53134: BlockingRecv error: recv got EOF from 127.0.0.1:53134 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 00:53:33.592605 16370 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:33.591223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53144 (local address 127.15.147.189:36055)
0806 00:53:33.591372 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.591377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.591675 (+ 298us) server_negotiation.cc:407] Connection header received
0806 00:53:33.591803 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.591806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.591873 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.591963 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.592494 (+ 531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53144: BlockingRecv error: recv got EOF from 127.0.0.1:53144 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 00:53:33.611460 16370 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:33.610282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53150 (local address 127.15.147.189:36055)
0806 00:53:33.610436 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.610441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.610592 (+ 151us) server_negotiation.cc:407] Connection header received
0806 00:53:33.610758 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.610761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.610825 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.610916 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.611345 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53150: BlockingRecv error: recv got EOF from 127.0.0.1:53150 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 00:53:33.646463 16370 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:33.645421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53152 (local address 127.15.147.189:36055)
0806 00:53:33.645568 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.645572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.645722 (+ 150us) server_negotiation.cc:407] Connection header received
0806 00:53:33.645842 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.645844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.645893 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.645973 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.646366 (+ 393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53152: BlockingRecv error: recv got EOF from 127.0.0.1:53152 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 00:53:33.713807 16370 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:33.712727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53162 (local address 127.15.147.189:36055)
0806 00:53:33.712878 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.712882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.713019 (+ 137us) server_negotiation.cc:407] Connection header received
0806 00:53:33.713143 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.713145 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.713193 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.713274 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.713708 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53162: BlockingRecv error: recv got EOF from 127.0.0.1:53162 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 00:53:33.845144 16370 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:33.844104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53178 (local address 127.15.147.189:36055)
0806 00:53:33.844249 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:33.844253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:33.844399 (+ 146us) server_negotiation.cc:407] Connection header received
0806 00:53:33.844518 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:33.844521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:33.844570 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:33.844658 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:33.845048 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53178: BlockingRecv error: recv got EOF from 127.0.0.1:53178 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 00:53:34.104621 16370 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 00:53:34.103402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53190 (local address 127.15.147.189:36055)
0806 00:53:34.103564 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:34.103569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:34.103679 (+ 110us) server_negotiation.cc:407] Connection header received
0806 00:53:34.103859 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:34.103863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:34.103918 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:34.104017 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 00:53:34.104525 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53190: BlockingRecv error: recv got EOF from 127.0.0.1:53190 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 00:53:34.385241 16546 catalog_manager.cc:1259] Loaded cluster ID: 7e5f2c68708440c283814da901a2368c
I20260806 00:53:34.385331 16546 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d: loading cluster ID for follower catalog manager: success
I20260806 00:53:34.387092 16546 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d: acquiring CA information for follower catalog manager: success
I20260806 00:53:34.388217 16546 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 66ffc2f6581744b19b2375a500195e8d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:53:34.392082 16550 catalog_manager.cc:1259] Loaded cluster ID: 7e5f2c68708440c283814da901a2368c
I20260806 00:53:34.392141 16550 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab: loading cluster ID for follower catalog manager: success
I20260806 00:53:34.393710 16550 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab: acquiring CA information for follower catalog manager: success
I20260806 00:53:34.394735 16550 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2955dd58f19745e69d87b228f8bf59ab: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 00:53:34.622957 16654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:34.618966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53202 (local address 127.15.147.189:36055)
0806 00:53:34.619220 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:34.619227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:34.619244 (+ 17us) server_negotiation.cc:407] Connection header received
0806 00:53:34.619437 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:34.619441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:34.619582 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:34.619702 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:34.620000 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:34.620513 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:34.621378 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:34.621622 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:34.621733 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:34.622097 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:34.622549 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:34.622694 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:34.622730 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 00:53:34.622796 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":66,"threads_started":1}
I20260806 00:53:34.629654 16661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:34.626083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40116 (local address 127.15.147.188:34409)
0806 00:53:34.626311 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:34.626315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:34.626328 (+ 13us) server_negotiation.cc:407] Connection header received
0806 00:53:34.626455 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:34.626457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:34.626566 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:34.626663 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:34.626896 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:34.627396 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:34.628148 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:34.628445 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:34.628526 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:34.628840 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:34.629217 (+ 377us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:34.629329 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:34.629398 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 00:53:34.629448 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":71,"threads_started":1}
I20260806 00:53:34.637255 16661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:34.632714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40120 (local address 127.15.147.188:34409)
0806 00:53:34.632847 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:34.632851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:34.633209 (+ 358us) server_negotiation.cc:407] Connection header received
0806 00:53:34.633327 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:34.633332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:34.633399 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:34.633468 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:34.634103 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:34.634580 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:34.635967 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:34.636209 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:34.636363 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:34.636758 (+ 395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:34.636831 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:34.636876 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:34.637050 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 00:53:34.637111 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 00:53:34.637275 16654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:34.632493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53214 (local address 127.15.147.189:36055)
0806 00:53:34.632631 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:34.632635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:34.633425 (+ 790us) server_negotiation.cc:407] Connection header received
0806 00:53:34.633534 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:34.633539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:34.633595 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:34.633680 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:34.633987 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:34.634736 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:34.635609 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:34.635859 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:34.635971 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:34.636791 (+ 820us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:34.636865 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:34.636951 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:34.637076 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 00:53:34.637125 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 00:53:34.637799 16668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 00:53:34.632425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48316 (local address 127.15.147.190:34027)
0806 00:53:34.632718 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0806 00:53:34.632723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 00:53:34.633040 (+ 317us) server_negotiation.cc:407] Connection header received
0806 00:53:34.633256 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 00:53:34.633259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 00:53:34.633438 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 00:53:34.633637 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 00:53:34.633850 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:34.634671 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:34.635988 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 00:53:34.636210 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 00:53:34.636345 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 00:53:34.636754 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 00:53:34.637069 (+ 315us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 00:53:34.637148 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 00:53:34.637593 (+ 445us) server_negotiation.cc:299] Negotiation successful
0806 00:53:34.637654 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":106,"threads_started":1}
I20260806 00:53:34.640890 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 16159
I20260806 00:53:34.657321 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 16266
I20260806 00:53:34.667376 15950 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3y4EW9/build/debug/bin/kudu with pid 16372
2026-08-06T00:53:34Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3324 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6570 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (297626 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST